chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,215 @@
|
||||
# Advanced Development Guide (File transcription service)
|
||||
|
||||
([简体中文](SDK_advanced_guide_offline_zh.md)|English)
|
||||
|
||||
[//]: # (FunASR provides a Chinese offline file transcription service that can be deployed locally or on a cloud server with just one click. The core of the service is the FunASR runtime SDK, which has been open-sourced. FunASR-runtime combines various capabilities such as speech endpoint detection (VAD), large-scale speech recognition (ASR) using Paraformer-large, and punctuation detection (PUNC), which have all been open-sourced by the speech laboratory of DAMO Academy on the Modelscope community. This enables accurate and efficient high-concurrency transcription of audio files.)
|
||||
FunASR Offline File Transcription Software Package provides a powerful speech-to-text offline file transcription service. With a complete speech recognition pipeline, it combines models for speech endpoint detection, speech recognition, punctuation, etc., allowing for the transcription of long audio and video files, spanning several hours, into punctuated text. It supports simultaneous transcription of hundreds of concurrent requests. The output is text with punctuation, including word-level timestamps, and it supports ITN (Initial Time Normalization) and user-defined hotwords. The server-side integration includes ffmpeg, enabling support for various audio and video formats as input. The software package provides client libraries in multiple programming languages such as HTML, Python, C++, Java, and C#, allowing users to use and further develop the software.
|
||||
|
||||
This document serves as a development guide for the FunASR offline file transcription service. If you wish to quickly experience the offline file transcription service, please refer to the one-click deployment example for the FunASR offline file transcription service ([docs](./SDK_tutorial.md)).
|
||||
|
||||
<img src="images/offline_structure.jpg" width="900"/>
|
||||
|
||||
|
||||
| TIME | INFO | IMAGE VERSION | IMAGE ID |
|
||||
|------------|----------------------------------------------------------------------------------------------------------------------------------|------------------------------|--------------|
|
||||
| 2024.09.26 | Fix memory leak, Support the SensevoiceSmall onnx model | funasr-runtime-sdk-cpu-0.4.6 | 8651c6b8a1ae |
|
||||
| 2024.05.15 | Adapting to FunASR 1.0 model structure | funasr-runtime-sdk-cpu-0.4.5 | 058b9882ae67 |
|
||||
| 2024.03.05 | docker image supports ARM64 platform, update modelscope | funasr-runtime-sdk-cpu-0.4.4 | 2dc87b86dc49 |
|
||||
| 2024.01.25 | Optimized the VAD (Voice Activity Detection) data processing method, significantly reducing peak memory usage; memory leak optimization| funasr-runtime-sdk-cpu-0.4.2 | befdc7b179ed |
|
||||
| 2024.01.08 | optimized format sentence-level timestamps | funasr-runtime-sdk-cpu-0.4.1 | 0250f8ef981b |
|
||||
| 2024.01.03 | Added support for 8k models, optimized timestamp mismatch issues and added sentence-level timestamps, improved the effectiveness of English word FST hotwords, supported automated configuration of thread parameters, and fixed known crash issues as well as memory leak problems. | funasr-runtime-sdk-cpu-0.4.0 | c4483ee08f04 |
|
||||
| 2023.11.08 | supporting punc-large model, Ngram model, fst hotwords, server-side loading of hotwords, adaptation to runtime structure changes | funasr-runtime-sdk-cpu-0.3.0 | caa64bddbb43 |
|
||||
| 2023.09.19 | supporting ITN model | funasr-runtime-sdk-cpu-0.2.2 | 2c5286be13e9 |
|
||||
| 2023.08.22 | integrated ffmpeg to support various audio and video inputs, supporting nn-hotword model and timestamp model | funasr-runtime-sdk-cpu-0.2.0 | 1ad3d19e0707 |
|
||||
| 2023.07.03 | 1.0 released | funasr-runtime-sdk-cpu-0.1.0 | 1ad3d19e0707 |
|
||||
|
||||
|
||||
## Quick start
|
||||
### Docker install
|
||||
If you have already installed Docker, ignore this step!
|
||||
```shell
|
||||
curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/install_docker.sh;
|
||||
sudo bash install_docker.sh
|
||||
```
|
||||
If you do not have Docker installed, please refer to [Docker Installation](https://alibaba-damo-academy.github.io/FunASR/en/installation/docker.html)
|
||||
|
||||
### Pulling and launching images
|
||||
Use the following command to pull and launch the Docker image for the FunASR runtime-SDK:
|
||||
```shell
|
||||
sudo docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-cpu-0.4.7
|
||||
|
||||
sudo docker run -p 10095:10095 -it --privileged=true -v /root:/workspace/models registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-cpu-0.4.7
|
||||
```
|
||||
|
||||
Introduction to command parameters:
|
||||
```text
|
||||
-p <host port>:<mapped docker port>: In the example, host machine (ECS) port 10095 is mapped to port 10095 in the Docker container. Make sure that port 10095 is open in the ECS security rules.
|
||||
|
||||
-v <host path>:<mounted Docker path>: In the example, the host machine path /root is mounted to the Docker path /workspace/models.
|
||||
```
|
||||
|
||||
### Starting the server
|
||||
Use the flollowing script to start the server ：
|
||||
```shell
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--hotword /workspace/models/hotwords.txt > log.txt 2>&1 &
|
||||
|
||||
# If you want to close ssl，please add：--certfile 0
|
||||
# If you want to deploy the timestamp or nn hotword model, please set --model-dir to the corresponding model:
|
||||
# damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx（timestamp）
|
||||
# damo/speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404-onnx（hotword）
|
||||
# If you want to load hotwords on the server side, please configure the hotwords in the host machine file ./funasr-runtime-resources/models/hotwords.txt (docker mapping address: /workspace/models/hotwords.txt):
|
||||
# One hotword per line, format (hotword weight): 阿里巴巴 20"
|
||||
```
|
||||
|
||||
### More details about the script run_server.sh:
|
||||
|
||||
The funasr-wss-server supports downloading models from Modelscope. You can set the model download address (--download-model-dir, default is /workspace/models) and the model ID (--model-dir, --vad-dir, --punc-dir). Here is an example:
|
||||
|
||||
```shell
|
||||
cd /workspace/FunASR/runtime
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--model-dir damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--certfile ../../../ssl_key/server.crt \
|
||||
--keyfile ../../../ssl_key/server.key \
|
||||
--hotword ../../hotwords.txt > log.txt 2>&1 &
|
||||
```
|
||||
|
||||
Introduction to run_server.sh parameters:
|
||||
```text
|
||||
--download-model-dir: Model download address, download models from Modelscope by setting the model ID.
|
||||
--model-dir: modelscope model ID or local model path.
|
||||
--vad-dir: modelscope model ID or local model path.
|
||||
--punc-dir: modelscope model ID or local model path.
|
||||
--itn-dir modelscope model ID or local model path.
|
||||
--port: Port number that the server listens on. Default is 10095.
|
||||
--decoder-thread-num: The number of thread pools on the server side that can handle concurrent requests.
|
||||
The script will automatically configure parameters decoder-thread-num and io-thread-num based on the server's thread count.
|
||||
--io-thread-num: Number of IO threads that the server starts.
|
||||
--model-thread-num: The number of internal threads for each recognition route to control the parallelism of the ONNX model.
|
||||
The default value is 1. It is recommended that decoder-thread-num * model-thread-num equals the total number of threads.
|
||||
--certfile <string>: SSL certificate file. Default is ../../../ssl_key/server.crt. If you want to close ssl，set 0
|
||||
--keyfile <string>: SSL key file. Default is ../../../ssl_key/server.key.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20), if the client provides hot words, then combined with the hot words provided by the client.
|
||||
```
|
||||
|
||||
### Shutting Down the FunASR Service
|
||||
```text
|
||||
# Check the PID of the funasr-wss-server process
|
||||
ps -x | grep funasr-wss-server
|
||||
kill -9 PID
|
||||
```
|
||||
|
||||
### Modifying Models and Other Parameters
|
||||
To replace the currently used model or other parameters, you need to first shut down the FunASR service, make the necessary modifications to the parameters you want to replace, and then restart the FunASR service. The model should be either an ASR/VAD/PUNC model from ModelScope or a fine-tuned model obtained from ModelScope.
|
||||
```text
|
||||
# For example, to replace the ASR model with damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx, use the following parameter setting --model-dir
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
# Set the port number using --port
|
||||
--port <port number>
|
||||
# Set the number of inference threads the server will start using --decoder-thread-num
|
||||
--decoder-thread-num <decoder thread num>
|
||||
# Set the number of IO threads the server will start using --io-thread-num
|
||||
--io-thread-num <io thread num>
|
||||
# Disable SSL certificate
|
||||
--certfile 0
|
||||
```
|
||||
|
||||
After executing the above command, the real-time speech transcription service will be started. If the model is specified as a ModelScope model id, the following models will be automatically downloaded from ModelScope:
|
||||
[FSMN-VAD](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary),
|
||||
[Paraformer-lagre](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx/summary),
|
||||
[CT-Transformer](https://www.modelscope.cn/models/damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx/summary),
|
||||
[FST-ITN](https://www.modelscope.cn/models/thuduj12/fst_itn_zh/summary),
|
||||
[Ngram lm](https://www.modelscope.cn/models/damo/speech_ngram_lm_zh-cn-ai-wesp-fst/summary)
|
||||
|
||||
If you wish to deploy your fine-tuned model (e.g., 10epoch.pb), you need to manually rename the model to model.pb and replace the original model.pb in ModelScope. Then, specify the path as `model_dir`.
|
||||
|
||||
## Starting the client
|
||||
After completing the deployment of FunASR offline file transcription service on the server, you can test and use the service by following these steps. Currently, FunASR-bin supports multiple ways to start the client. The following are command-line examples based on python-client, c++-client, and custom client Websocket communication protocol:
|
||||
|
||||
### python-client
|
||||
```shell
|
||||
python funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "./data/wav.scp" --send_without_sleep --output_dir "./results"
|
||||
```
|
||||
Introduction to command parameters:
|
||||
```text
|
||||
--host: the IP address of the server. It can be set to 127.0.0.1 for local testing.
|
||||
--port: the port number of the server listener.
|
||||
--audio_in: the audio input. Input can be a path to a wav file or a wav.scp file (a Kaldi-formatted wav list in which each line includes a wav_id followed by a tab and a wav_path).
|
||||
--output_dir: the path to the recognition result output.
|
||||
--ssl: whether to use SSL encryption. The default is to use SSL.
|
||||
--mode: offline mode.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use_itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### c++-client
|
||||
```shell
|
||||
. /funasr-wss-client --server-ip 127.0.0.1 --port 10095 --wav-path test.wav --thread-num 1 --is-ssl 1
|
||||
```
|
||||
|
||||
Introduction to command parameters:
|
||||
```text
|
||||
--server-ip: the IP address of the server. It can be set to 127.0.0.1 for local testing.
|
||||
--port: the port number of the server listener.
|
||||
--wav-path: the audio input. Input can be a path to a wav file or a wav.scp file (a Kaldi-formatted wav list in which each line includes a wav_id followed by a tab and a wav_path).
|
||||
--is-ssl: whether to use SSL encryption. The default is to use SSL.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use-itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### Custom client
|
||||
If you want to define your own client, see the [Websocket communication protocol](./websocket_protocol.md)
|
||||
|
||||
## How to customize service deployment
|
||||
The code for FunASR-runtime is open source. If the server and client cannot fully meet your needs, you can further develop them based on your own requirements:
|
||||
|
||||
### C++ client
|
||||
https://github.com/modelscope/FunASR/tree/main/runtime/websocket
|
||||
|
||||
### Python client
|
||||
https://github.com/modelscope/FunASR/tree/main/runtime/python/websocket
|
||||
|
||||
### C++ server
|
||||
|
||||
#### VAD
|
||||
```c++
|
||||
// The use of the VAD model consists of two steps: FsmnVadInit and FsmnVadInfer:
|
||||
FUNASR_HANDLE vad_hanlde=FsmnVadInit(model_path, thread_num);
|
||||
// Where: model_path contains "model-dir" and "quantize", thread_num is the ONNX thread count;
|
||||
FUNASR_RESULT result=FsmnVadInfer(vad_hanlde, wav_file.c_str(), NULL, 16000);
|
||||
// Where: vad_hanlde is the return value of FunOfflineInit, wav_file is the path to the audio file, and sampling_rate is the sampling rate (default 16k).
|
||||
```
|
||||
|
||||
See the usage example for details [docs](https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline-vad.cpp)
|
||||
|
||||
#### ASR
|
||||
```text
|
||||
// The use of the ASR model consists of two steps: FunOfflineInit and FunOfflineInfer:
|
||||
FUNASR_HANDLE asr_hanlde=FunOfflineInit(model_path, thread_num);
|
||||
// Where: model_path contains "model-dir" and "quantize", thread_num is the ONNX thread count;
|
||||
FUNASR_RESULT result=FunOfflineInfer(asr_hanlde, wav_file.c_str(), RASR_NONE, NULL, 16000);
|
||||
// Where: asr_hanlde is the return value of FunOfflineInit, wav_file is the path to the audio file, and sampling_rate is the sampling rate (default 16k).
|
||||
```
|
||||
See the usage example for details, [docs](https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline.cpp)
|
||||
|
||||
#### PUNC
|
||||
```text
|
||||
// The use of the PUNC model consists of two steps: CTTransformerInit and CTTransformerInfer:
|
||||
FUNASR_HANDLE punc_hanlde=CTTransformerInit(model_path, thread_num);
|
||||
// Where: model_path contains "model-dir" and "quantize", thread_num is the ONNX thread count;
|
||||
FUNASR_RESULT result=CTTransformerInfer(punc_hanlde, txt_str.c_str(), RASR_NONE, NULL);
|
||||
// Where: punc_hanlde is the return value of CTTransformerInit, txt_str is the text
|
||||
```
|
||||
See the usage example for details, [docs](https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline-punc.cpp)
|
||||
@@ -0,0 +1,199 @@
|
||||
# Advanced Development Guide (File transcription service)
|
||||
|
||||
FunASR provides a English offline file transcription service that can be deployed locally or on a cloud server with just one click. The core of the service is the FunASR runtime SDK, which has been open-sourced. FunASR-runtime combines various capabilities such as speech endpoint detection (VAD), large-scale speech recognition (ASR) using Paraformer-large, and punctuation detection (PUNC), which have all been open-sourced by the speech laboratory of DAMO Academy on the Modelscope community. This enables accurate and efficient high-concurrency transcription of audio files.
|
||||
|
||||
This document serves as a development guide for the FunASR offline file transcription service. If you wish to quickly experience the offline file transcription service, please refer to the one-click deployment example for the FunASR offline file transcription service ([docs](./SDK_tutorial.md)).
|
||||
|
||||
| TIME | INFO | IMAGE VERSION | IMAGE ID |
|
||||
|------------|-----------------------------------------|---------------------------------|--------------|
|
||||
| 2024.09.26 | Fix memory leak | funasr-runtime-sdk-en-cpu-0.1.7 | f6c5a7b59eb6 |
|
||||
| 2024.05.15 | Adapting to FunASR 1.0 model structure | funasr-runtime-sdk-en-cpu-0.1.6 | 84d781d07997 |
|
||||
| 2024.03.05 | docker image supports ARM64 platform, update modelscope | funasr-runtime-sdk-en-cpu-0.1.5 | 7cca2abc5901 |
|
||||
| 2024.01.25 | Optimized the VAD (Voice Activity Detection) data processing method, significantly reducing peak memory usage; memory leak optimization| funasr-runtime-sdk-en-cpu-0.1.3 | c00f9ce7a195 |
|
||||
| 2024.01.03 | fixed known crash issues as well as memory leak problems | funasr-runtime-sdk-en-cpu-0.1.2 | 0cdd9f4a4bb5 |
|
||||
| 2023.11.08 | Adaptation to runtime structure changes | funasr-runtime-sdk-en-cpu-0.1.1 | 27017f70f72a |
|
||||
| 2023.10.16 | 1.0 released | funasr-runtime-sdk-en-cpu-0.1.0 | e0de03eb0163 |
|
||||
|
||||
## Quick start
|
||||
### Docker install
|
||||
If you have already installed Docker, ignore this step!
|
||||
```shell
|
||||
curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/install_docker.sh;
|
||||
sudo bash install_docker.sh
|
||||
```
|
||||
If you do not have Docker installed, please refer to [Docker Installation](https://alibaba-damo-academy.github.io/FunASR/en/installation/docker.html)
|
||||
|
||||
### Pulling and launching images
|
||||
Use the following command to pull and launch the Docker image for the FunASR runtime-SDK:
|
||||
```shell
|
||||
sudo docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-en-cpu-0.1.8
|
||||
|
||||
sudo docker run -p 10097:10095 -it --privileged=true -v /root:/workspace/models registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-en-cpu-0.1.8
|
||||
```
|
||||
Introduction to command parameters:
|
||||
```text
|
||||
-p <host port>:<mapped docker port>: In the example, host machine (ECS) port 10097 is mapped to port 10095 in the Docker container. Make sure that port 10097 is open in the ECS security rules.
|
||||
|
||||
-v <host path>:<mounted Docker path>: In the example, the host machine path /root is mounted to the Docker path /workspace/models.
|
||||
|
||||
```
|
||||
|
||||
### Starting the server
|
||||
Use the flollowing script to start the server ：
|
||||
```shell
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-en-16k-common-vocab10020-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx > log.txt 2>&1 &
|
||||
|
||||
# If you want to close ssl，please add：--certfile 0
|
||||
```
|
||||
|
||||
### More details about the script run_server.sh:
|
||||
The funasr-wss-server supports downloading models from Modelscope. You can set the model download address (--download-model-dir, default is /workspace/models) and the model ID (--model-dir, --vad-dir, --punc-dir). Here is an example:
|
||||
|
||||
```shell
|
||||
cd /workspace/FunASR/runtime
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-en-16k-common-vocab10020-onnx \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--certfile ../../../ssl_key/server.crt \
|
||||
--keyfile ../../../ssl_key/server.key > log.txt 2>&1 &
|
||||
```
|
||||
|
||||
Introduction to run_server.sh parameters:
|
||||
```text
|
||||
--download-model-dir: Model download address, download models from Modelscope by setting the model ID.
|
||||
--model-dir: modelscope model ID or local model path.
|
||||
--vad-dir: modelscope model ID or local model path.
|
||||
--punc-dir: modelscope model ID or local model path.
|
||||
--itn-dir modelscope model ID or local model path.
|
||||
--port: Port number that the server listens on. Default is 10095.
|
||||
--decoder-thread-num: The number of thread pools on the server side that can handle concurrent requests.
|
||||
The script will automatically configure parameters decoder-thread-num and io-thread-num based on the server's thread count.
|
||||
--io-thread-num: Number of IO threads that the server starts.
|
||||
--model-thread-num: The number of internal threads for each recognition route to control the parallelism of the ONNX model.
|
||||
The default value is 1. It is recommended that decoder-thread-num * model-thread-num equals the total number of threads.
|
||||
--certfile <string>: SSL certificate file. Default is ../../../ssl_key/server.crt. If you want to close ssl，set 0
|
||||
--keyfile <string>: SSL key file. Default is ../../../ssl_key/server.key.
|
||||
```
|
||||
|
||||
### Shutting Down the FunASR Service
|
||||
```text
|
||||
# Check the PID of the funasr-wss-server process
|
||||
ps -x | grep funasr-wss-server
|
||||
kill -9 PID
|
||||
```
|
||||
|
||||
### Modifying Models and Other Parameters
|
||||
To replace the currently used model or other parameters, you need to first shut down the FunASR service, make the necessary modifications to the parameters you want to replace, and then restart the FunASR service. The model should be either an ASR/VAD/PUNC model from ModelScope or a fine-tuned model obtained from ModelScope.
|
||||
```text
|
||||
# For example, to replace the ASR model with damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx, use the following parameter setting --model-dir
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
# Set the port number using --port
|
||||
--port <port number>
|
||||
# Set the number of inference threads the server will start using --decoder-thread-num
|
||||
--decoder-thread-num <decoder thread num>
|
||||
# Set the number of IO threads the server will start using --io-thread-num
|
||||
--io-thread-num <io thread num>
|
||||
# Disable SSL certificate
|
||||
--certfile 0
|
||||
```
|
||||
|
||||
After executing the above command, the real-time speech transcription service will be started. If the model is specified as a ModelScope model id, the following models will be automatically downloaded from ModelScope:
|
||||
[FSMN-VAD](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary),
|
||||
[Paraformer-lagre](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-en-16k-common-vocab10020-onnx/summary),
|
||||
[CT-Transformer](https://www.modelscope.cn/models/damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx/summary)
|
||||
|
||||
If you wish to deploy your fine-tuned model (e.g., 10epoch.pb), you need to manually rename the model to model.pb and replace the original model.pb in ModelScope. Then, specify the path as `model_dir`.
|
||||
|
||||
## Starting the client
|
||||
|
||||
After completing the deployment of FunASR offline file transcription service on the server, you can test and use the service by following these steps. Currently, FunASR-bin supports multiple ways to start the client. The following are command-line examples based on python-client, c++-client, and custom client Websocket communication protocol:
|
||||
|
||||
### python-client
|
||||
```shell
|
||||
python funasr_wss_client.py --host "127.0.0.1" --port 10097 --mode offline --audio_in "./data/wav.scp" --send_without_sleep --output_dir "./results"
|
||||
```
|
||||
|
||||
Introduction to command parameters:
|
||||
|
||||
```text
|
||||
--host: the IP address of the server. It can be set to 127.0.0.1 for local testing.
|
||||
--port: the port number of the server listener.
|
||||
--audio_in: the audio input. Input can be a path to a wav file or a wav.scp file (a Kaldi-formatted wav list in which each line includes a wav_id followed by a tab and a wav_path).
|
||||
--output_dir: the path to the recognition result output.
|
||||
--ssl: whether to use SSL encryption. The default is to use SSL.
|
||||
--mode: offline mode.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use_itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### c++-client
|
||||
```shell
|
||||
. /funasr-wss-client --server-ip 127.0.0.1 --port 10097 --wav-path test.wav --thread-num 1 --is-ssl 1
|
||||
```
|
||||
|
||||
Introduction to command parameters:
|
||||
|
||||
```text
|
||||
--server-ip: the IP address of the server. It can be set to 127.0.0.1 for local testing.
|
||||
--port: the port number of the server listener.
|
||||
--wav-path: the audio input. Input can be a path to a wav file or a wav.scp file (a Kaldi-formatted wav list in which each line includes a wav_id followed by a tab and a wav_path).
|
||||
--is-ssl: whether to use SSL encryption. The default is to use SSL.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use-itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### Custom client
|
||||
|
||||
If you want to define your own client, see the [Websocket communication protocol](./websocket_protocol.md)
|
||||
|
||||
## How to customize service deployment
|
||||
|
||||
The code for FunASR-runtime is open source. If the server and client cannot fully meet your needs, you can further develop them based on your own requirements:
|
||||
|
||||
### C++ client
|
||||
|
||||
https://github.com/modelscope/FunASR/tree/main/runtime/websocket
|
||||
|
||||
### Python client
|
||||
|
||||
https://github.com/modelscope/FunASR/tree/main/runtime/python/websocket
|
||||
|
||||
### C++ server
|
||||
|
||||
#### VAD
|
||||
```c++
|
||||
// The use of the VAD model consists of two steps: FsmnVadInit and FsmnVadInfer:
|
||||
FUNASR_HANDLE vad_hanlde=FsmnVadInit(model_path, thread_num);
|
||||
// Where: model_path contains "model-dir" and "quantize", thread_num is the ONNX thread count;
|
||||
FUNASR_RESULT result=FsmnVadInfer(vad_hanlde, wav_file.c_str(), NULL, 16000);
|
||||
// Where: vad_hanlde is the return value of FunOfflineInit, wav_file is the path to the audio file, and sampling_rate is the sampling rate (default 16k).
|
||||
```
|
||||
|
||||
See the usage example for details [docs](https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline-vad.cpp)
|
||||
|
||||
#### ASR
|
||||
```text
|
||||
// The use of the ASR model consists of two steps: FunOfflineInit and FunOfflineInfer:
|
||||
FUNASR_HANDLE asr_hanlde=FunOfflineInit(model_path, thread_num);
|
||||
// Where: model_path contains "model-dir" and "quantize", thread_num is the ONNX thread count;
|
||||
FUNASR_RESULT result=FunOfflineInfer(asr_hanlde, wav_file.c_str(), RASR_NONE, NULL, 16000);
|
||||
// Where: asr_hanlde is the return value of FunOfflineInit, wav_file is the path to the audio file, and sampling_rate is the sampling rate (default 16k).
|
||||
```
|
||||
|
||||
See the usage example for details, [docs](https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline.cpp)
|
||||
|
||||
#### PUNC
|
||||
```text
|
||||
// The use of the PUNC model consists of two steps: CTTransformerInit and CTTransformerInfer:
|
||||
FUNASR_HANDLE punc_hanlde=CTTransformerInit(model_path, thread_num);
|
||||
// Where: model_path contains "model-dir" and "quantize", thread_num is the ONNX thread count;
|
||||
FUNASR_RESULT result=CTTransformerInfer(punc_hanlde, txt_str.c_str(), RASR_NONE, NULL);
|
||||
// Where: punc_hanlde is the return value of CTTransformerInit, txt_str is the text
|
||||
```
|
||||
See the usage example for details, [docs](https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline-punc.cpp)
|
||||
@@ -0,0 +1,245 @@
|
||||
# FunASR英文离线文件转写服务开发指南
|
||||
|
||||
FunASR提供可一键本地或者云端服务器部署的英文离线文件转写服务，内核为FunASR已开源runtime-SDK。FunASR-runtime结合了达摩院语音实验室在Modelscope社区开源的语音端点检测(VAD)、Paraformer-large语音识别(ASR)、标点检测(PUNC) 等相关能力，可以准确、高效的对音频进行高并发转写。
|
||||
|
||||
本文档为FunASR离线文件转写服务开发指南。如果您想快速体验离线文件转写服务，可参考[快速上手](#快速上手)。
|
||||
|
||||
| 时间 | 详情 | 镜像版本 | 镜像ID |
|
||||
|------------|---------------|---------------------------------|--------------|
|
||||
| 2024.09.26 | 修复内存泄漏 | funasr-runtime-sdk-en-cpu-0.1.7 | f6c5a7b59eb6 |
|
||||
| 2024.05.15 | 适配FunASR 1.0模型结构 | funasr-runtime-sdk-en-cpu-0.1.6 | 84d781d07997 |
|
||||
| 2024.03.05 | docker镜像支持arm64平台，升级modelscope版本 | funasr-runtime-sdk-en-cpu-0.1.5 | 7cca2abc5901 |
|
||||
| 2024.01.25 | 优化vad数据处理方式，大幅降低峰值内存占用；内存泄漏优化 | funasr-runtime-sdk-en-cpu-0.1.3 | c00f9ce7a195 |
|
||||
| 2024.01.03 | 修复已知的crash问题及内存泄漏问题 | funasr-runtime-sdk-en-cpu-0.1.2 | 0cdd9f4a4bb5 |
|
||||
| 2023.11.08 | runtime结构变化适配 | funasr-runtime-sdk-en-cpu-0.1.1 | 27017f70f72a |
|
||||
| 2023.10.16 | 1.0 发布 | funasr-runtime-sdk-en-cpu-0.1.0 | e0de03eb0163 |
|
||||
|
||||
|
||||
## 服务器配置
|
||||
|
||||
用户可以根据自己的业务需求，选择合适的服务器配置，推荐配置为：
|
||||
- 配置1: （X86，计算型），4核vCPU，内存8G，单机可以支持大约32路的请求
|
||||
- 配置2: （X86，计算型），16核vCPU，内存32G，单机可以支持大约64路的请求
|
||||
- 配置3: （X86，计算型），64核vCPU，内存128G，单机可以支持大约200路的请求
|
||||
|
||||
详细性能测试报告（[点击此处](./benchmark_onnx_cpp.md)）
|
||||
|
||||
云服务厂商，针对新用户，有3个月免费试用活动，申请教程（[点击此处](https://github.com/modelscope/FunASR/blob/main/runtime/docs/aliyun_server_tutorial.md)）
|
||||
|
||||
|
||||
## 快速上手
|
||||
### docker安装
|
||||
如果您已安装docker，忽略本步骤！!
|
||||
通过下述命令在服务器上安装docker：
|
||||
```shell
|
||||
curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/install_docker.sh；
|
||||
sudo bash install_docker.sh
|
||||
```
|
||||
docker安装失败请参考 [Docker Installation](https://alibaba-damo-academy.github.io/FunASR/en/installation/docker.html)
|
||||
|
||||
### 镜像启动
|
||||
通过下述命令拉取并启动FunASR runtime-SDK的docker镜像：
|
||||
```shell
|
||||
sudo docker pull \
|
||||
registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-en-cpu-0.1.8
|
||||
mkdir -p ./funasr-runtime-resources/models
|
||||
sudo docker run -p 10097:10095 -it --privileged=true \
|
||||
-v $PWD/funasr-runtime-resources/models:/workspace/models \
|
||||
registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-en-cpu-0.1.8
|
||||
```
|
||||
|
||||
### 服务端启动
|
||||
|
||||
docker启动之后，启动 funasr-wss-server服务程序：
|
||||
```shell
|
||||
cd FunASR/runtime
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-en-16k-common-vocab10020-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx > log.txt 2>&1 &
|
||||
|
||||
# 如果您想关闭ssl，增加参数：--certfile 0
|
||||
|
||||
```
|
||||
服务端详细参数介绍可参考[服务端用法详解](#服务端用法详解)
|
||||
### 客户端测试与使用
|
||||
|
||||
下载客户端测试工具目录samples
|
||||
```shell
|
||||
wget https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz
|
||||
```
|
||||
我们以Python语言客户端为例，进行说明，支持多种音频格式输入（.wav, .pcm, .mp3等），也支持视频输入(.mp4等)，以及多文件列表wav.scp输入，其他版本客户端请参考文档（[点击此处](#客户端用法详解)），定制服务部署请参考[如何定制服务部署](#如何定制服务部署)
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10097 --mode offline --audio_in "../audio/asr_example.wav"
|
||||
```
|
||||
|
||||
------------------
|
||||
## 客户端用法详解
|
||||
|
||||
在服务器上完成FunASR服务部署以后，可以通过如下的步骤来测试和使用离线文件转写服务。
|
||||
目前分别支持以下几种编程语言客户端
|
||||
|
||||
- [Python](#python-client)
|
||||
- [CPP](#cpp-client)
|
||||
- [html网页版本](#Html网页版)
|
||||
- [Java](#Java-client)
|
||||
|
||||
### python-client
|
||||
若想直接运行client进行测试，可参考如下简易说明，以python版本为例：
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10097 --mode offline \
|
||||
--audio_in "../audio/asr_example.wav" --output_dir "./results"
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
```text
|
||||
--host 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10097 部署端口号
|
||||
--mode offline表示离线文件转写
|
||||
--audio_in 需要进行转写的音频文件，支持文件路径，文件列表wav.scp
|
||||
--thread_num 设置并发发送线程数，默认为1
|
||||
--ssl 设置是否开启ssl证书校验，默认1开启，设置为0关闭
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--use_itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### cpp-client
|
||||
进入samples/cpp目录后，可以用cpp进行测试，指令如下：
|
||||
```shell
|
||||
./funasr-wss-client --server-ip 127.0.0.1 --port 10097 --wav-path ../audio/asr_example.wav
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
|
||||
```text
|
||||
--server-ip 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10097 部署端口号
|
||||
--wav-path 需要进行转写的音频文件，支持文件路径
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--use-itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### Html网页版
|
||||
|
||||
在浏览器中打开 html/static/index.html，即可出现如下页面，支持麦克风输入与文件上传，直接进行体验
|
||||
|
||||
<img src="images/html.png" width="900"/>
|
||||
|
||||
### Java-client
|
||||
|
||||
```shell
|
||||
FunasrWsClient --host localhost --port 10097 --audio_in ./asr_example.wav --mode offline
|
||||
```
|
||||
详细可以参考文档（[点击此处](../java/readme.md)）
|
||||
|
||||
## 服务端用法详解：
|
||||
|
||||
### 启动FunASR服务
|
||||
```shell
|
||||
cd /workspace/FunASR/runtime
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-en-16k-common-vocab10020-onnx \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--certfile ../../../ssl_key/server.crt \
|
||||
--keyfile ../../../ssl_key/server.key > log.txt 2>&1 &
|
||||
```
|
||||
**run_server.sh命令参数介绍**
|
||||
```text
|
||||
--download-model-dir 模型下载地址，通过设置model ID从Modelscope下载模型
|
||||
--model-dir modelscope model ID 或者 本地模型路径
|
||||
--vad-dir modelscope model ID 或者 本地模型路径
|
||||
--punc-dir modelscope model ID 或者 本地模型路径
|
||||
--itn-dir modelscope model ID 或者 本地模型路径
|
||||
--port 服务端监听的端口号，默认为 10095
|
||||
--decoder-thread-num 服务端线程池个数(支持的最大并发路数)，
|
||||
脚本会根据服务器线程数自动配置decoder-thread-num、io-thread-num
|
||||
--io-thread-num 服务端启动的IO线程数
|
||||
--model-thread-num 每路识别的内部线程数(控制ONNX模型的并行)，默认为 1，
|
||||
其中建议 decoder-thread-num*model-thread-num 等于总线程数
|
||||
--certfile ssl的证书文件，默认为：../../../ssl_key/server.crt，如果需要关闭ssl，参数设置为0
|
||||
--keyfile ssl的密钥文件，默认为：../../../ssl_key/server.key
|
||||
```
|
||||
|
||||
### 关闭FunASR服务
|
||||
```text
|
||||
# 查看 funasr-wss-server 对应的PID
|
||||
ps -x | grep funasr-wss-server
|
||||
kill -9 PID
|
||||
```
|
||||
|
||||
### 修改模型及其他参数
|
||||
替换正在使用的模型或者其他参数，需先关闭FunASR服务，修改需要替换的参数，并重新启动FunASR服务。其中模型需为ModelScope中的ASR/VAD/PUNC模型，或者从ModelScope中模型finetune后的模型。
|
||||
```text
|
||||
# 例如替换ASR模型为 damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx，则如下设置参数 --model-dir
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
# 设置端口号 --port
|
||||
--port <port number>
|
||||
# 设置服务端启动的推理线程数 --decoder-thread-num
|
||||
--decoder-thread-num <decoder thread num>
|
||||
# 设置服务端启动的IO线程数 --io-thread-num
|
||||
--io-thread-num <io thread num>
|
||||
# 关闭SSL证书
|
||||
--certfile 0
|
||||
```
|
||||
|
||||
执行上述指令后，启动英文离线文件转写服务。如果模型指定为ModelScope中model id，会自动从MoldeScope中下载如下模型：
|
||||
[FSMN-VAD模型](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary),
|
||||
[Paraformer-lagre模型](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-en-16k-common-vocab10020-onnx/summary),
|
||||
[CT-Transformer标点预测模型](https://www.modelscope.cn/models/damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx/summary)
|
||||
|
||||
如果，您希望部署您finetune后的模型（例如10epoch.pb），需要手动将模型重命名为model.pb，并将原modelscope中模型model.pb替换掉，将路径指定为`model_dir`即可。
|
||||
|
||||
## 如何定制服务部署
|
||||
|
||||
FunASR-runtime的代码已开源，如果服务端和客户端不能很好的满足您的需求，您可以根据自己的需求进行进一步的开发：
|
||||
### c++ 客户端：
|
||||
|
||||
https://github.com/modelscope/FunASR/tree/main/runtime/websocket
|
||||
|
||||
### python 客户端：
|
||||
|
||||
https://github.com/modelscope/FunASR/tree/main/runtime/python/websocket
|
||||
|
||||
### 自定义客户端：
|
||||
|
||||
如果您想定义自己的client，参考[websocket通信协议](./websocket_protocol_zh.md)
|
||||
|
||||
### c++ 服务端：
|
||||
|
||||
#### VAD
|
||||
```c++
|
||||
// VAD模型的使用分为FsmnVadInit和FsmnVadInfer两个步骤：
|
||||
FUNASR_HANDLE vad_hanlde=FsmnVadInit(model_path, thread_num);
|
||||
// 其中：model_path 包含"model-dir"、"quantize"，thread_num为onnx线程数；
|
||||
FUNASR_RESULT result=FsmnVadInfer(vad_hanlde, wav_file.c_str(), NULL, 16000);
|
||||
// 其中：vad_hanlde为FunOfflineInit返回值，wav_file为音频路径，sampling_rate为采样率(默认16k)
|
||||
```
|
||||
|
||||
使用示例详见：https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline-vad.cpp
|
||||
|
||||
#### ASR
|
||||
```text
|
||||
// ASR模型的使用分为FunOfflineInit和FunOfflineInfer两个步骤：
|
||||
FUNASR_HANDLE asr_hanlde=FunOfflineInit(model_path, thread_num);
|
||||
// 其中：model_path 包含"model-dir"、"quantize"，thread_num为onnx线程数；
|
||||
FUNASR_RESULT result=FunOfflineInfer(asr_hanlde, wav_file.c_str(), RASR_NONE, NULL, 16000);
|
||||
// 其中：asr_hanlde为FunOfflineInit返回值，wav_file为音频路径，sampling_rate为采样率(默认16k)
|
||||
```
|
||||
|
||||
使用示例详见：https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline.cpp
|
||||
|
||||
#### PUNC
|
||||
```text
|
||||
// PUNC模型的使用分为CTTransformerInit和CTTransformerInfer两个步骤：
|
||||
FUNASR_HANDLE punc_hanlde=CTTransformerInit(model_path, thread_num);
|
||||
// 其中：model_path 包含"model-dir"、"quantize"，thread_num为onnx线程数；
|
||||
FUNASR_RESULT result=CTTransformerInfer(punc_hanlde, txt_str.c_str(), RASR_NONE, NULL);
|
||||
// 其中：punc_hanlde为CTTransformerInit返回值，txt_str为文本
|
||||
```
|
||||
使用示例详见：https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline-punc.cpp
|
||||
@@ -0,0 +1,176 @@
|
||||
# Advanced Development Guide (File transcription service GPU)
|
||||
|
||||
([简体中文](SDK_advanced_guide_offline_gpu_zh.md)|English)
|
||||
|
||||
[//]: # (FunASR provides a Chinese offline file transcription service that can be deployed locally or on a cloud server with just one click. The core of the service is the FunASR runtime SDK, which has been open-sourced. FunASR-runtime combines various capabilities such as speech endpoint detection (VAD), large-scale speech recognition (ASR) using Paraformer-large, and punctuation detection (PUNC), which have all been open-sourced by the speech laboratory of DAMO Academy on the Modelscope community. This enables accurate and efficient high-concurrency transcription of audio files.)
|
||||
FunASR Offline File Transcription Software Package(GPU) provides a powerful speech-to-text offline file transcription service. With a complete speech recognition pipeline, it combines models for speech endpoint detection, speech recognition, punctuation, etc., allowing for the transcription of long audio and video files, spanning several hours, into punctuated text. It supports simultaneous transcription of hundreds of concurrent requests. The output is text with punctuation, including word-level timestamps, and it supports ITN (Initial Time Normalization) and user-defined hotwords. The server-side integration includes ffmpeg, enabling support for various audio and video formats as input. The software package provides client libraries in multiple programming languages such as HTML, Python, C++, Java, and C#, allowing users to use and further develop the software.
|
||||
|
||||
This document serves as a development guide for the FunASR offline file transcription service. If you wish to quickly experience the offline file transcription service, please refer to the one-click deployment example for the FunASR offline file transcription service ([docs](./SDK_tutorial.md)).
|
||||
|
||||
<img src="images/offline_structure.jpg" width="900"/>
|
||||
|
||||
|
||||
| TIME | INFO | IMAGE VERSION | IMAGE ID |
|
||||
|------------|----------------------------------------------------------------------------------------------------------------------------------|------------------------------|--------------|
|
||||
| 2024.09.26 | Fix GPU memory leak | funasr-runtime-sdk-gpu-0.2.0 | d280bf7e495b |
|
||||
| 2024.07.01 | Optimize BladeDISC model compatibility issues | funasr-runtime-sdk-gpu-0.1.1 | 8875cbf9b99e |
|
||||
| 2024.06.27 | Offline File Transcription Software Package(GPU) 1.0 released | funasr-runtime-sdk-gpu-0.1.0 | b86066f4d018 |
|
||||
|
||||
|
||||
## Quick start
|
||||
### Docker install
|
||||
If you have already installed Docker, ignore this step!
|
||||
```shell
|
||||
curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/install_docker.sh;
|
||||
sudo bash install_docker.sh
|
||||
```
|
||||
If you do not have Docker installed, please refer to [Docker Installation](https://alibaba-damo-academy.github.io/FunASR/en/installation/docker.html)
|
||||
|
||||
### Pulling and launching images
|
||||
Use the following command to pull and launch the Docker image for the FunASR runtime-SDK:
|
||||
```shell
|
||||
sudo docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-gpu-0.2.1
|
||||
|
||||
sudo docker run --gpus=all -p 10098:10095 -it --privileged=true -v /root:/workspace/models registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-gpu-0.2.1
|
||||
```
|
||||
|
||||
Introduction to command parameters:
|
||||
```text
|
||||
-p <host port>:<mapped docker port>: In the example, host machine (ECS) port 10098 is mapped to port 10095 in the Docker container. Make sure that port 10098 is open in the ECS security rules.
|
||||
|
||||
-v <host path>:<mounted Docker path>: In the example, the host machine path /root is mounted to the Docker path /workspace/models.
|
||||
```
|
||||
|
||||
### Starting the server
|
||||
Use the flollowing script to start the server ：
|
||||
```shell
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--model-dir damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch \
|
||||
--punc-dir damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--hotword /workspace/models/hotwords.txt > log.txt 2>&1 &
|
||||
|
||||
***When the service starts for the first time, it will export the TorchScript model, which may take some time. Please be patient***
|
||||
# If you want to close ssl，please add：--certfile 0
|
||||
# If you want to deploy the timestamp or nn hotword model, please set --model-dir to the corresponding model:
|
||||
# damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch（timestamp）
|
||||
# damo/speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404（hotword）
|
||||
# If you want to load hotwords on the server side, please configure the hotwords in the host machine file ./funasr-runtime-resources/models/hotwords.txt (docker mapping address: /workspace/models/hotwords.txt):
|
||||
# One hotword per line, format (hotword weight): 阿里巴巴 20"
|
||||
```
|
||||
|
||||
### More details about the script run_server.sh:
|
||||
|
||||
The funasr-wss-server supports downloading models from Modelscope. You can set the model download address (--download-model-dir, default is /workspace/models) and the model ID (--model-dir, --vad-dir, --punc-dir). Here is an example:
|
||||
|
||||
```shell
|
||||
cd /workspace/FunASR/runtime
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--model-dir damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--certfile ../../../ssl_key/server.crt \
|
||||
--keyfile ../../../ssl_key/server.key \
|
||||
--hotword ../../hotwords.txt > log.txt 2>&1 &
|
||||
```
|
||||
|
||||
Introduction to run_server.sh parameters:
|
||||
```text
|
||||
--download-model-dir: Model download address, download models from Modelscope by setting the model ID.
|
||||
--model-dir: modelscope model ID or local model path.
|
||||
--vad-dir: modelscope model ID or local model path.
|
||||
--punc-dir: modelscope model ID or local model path.
|
||||
--itn-dir modelscope model ID or local model path.
|
||||
--port: Port number that the server listens on. Default is 10095.
|
||||
--decoder-thread-num: The number of thread pools on the server side that can handle concurrent requests.
|
||||
--io-thread-num: Number of IO threads that the server starts.
|
||||
--model-thread-num: The number of internal threads for each recognition route to control the parallelism of the ONNX model.
|
||||
The default value is 1. It is recommended that decoder-thread-num * model-thread-num equals the total number of threads.
|
||||
--certfile <string>: SSL certificate file. Default is ../../../ssl_key/server.crt. If you want to close ssl，set 0
|
||||
--keyfile <string>: SSL key file. Default is ../../../ssl_key/server.key.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20), if the client provides hot words, then combined with the hot words provided by the client.
|
||||
```
|
||||
|
||||
### Shutting Down the FunASR Service
|
||||
```text
|
||||
# Check the PID of the funasr-wss-server process
|
||||
ps -x | grep funasr-wss-server
|
||||
kill -9 PID
|
||||
```
|
||||
|
||||
### Modifying Models and Other Parameters
|
||||
To replace the currently used model or other parameters, you need to first shut down the FunASR service, make the necessary modifications to the parameters you want to replace, and then restart the FunASR service. The model should be either an ASR/VAD/PUNC model from ModelScope or a fine-tuned model obtained from ModelScope.
|
||||
```text
|
||||
# For example, to replace the ASR model with damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch, use the following parameter setting --model-dir
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch
|
||||
# Set the port number using --port
|
||||
--port <port number>
|
||||
# Set the number of inference threads the server will start using --decoder-thread-num
|
||||
--decoder-thread-num <decoder thread num>
|
||||
# Set the number of IO threads the server will start using --io-thread-num
|
||||
--io-thread-num <io thread num>
|
||||
# Disable SSL certificate
|
||||
--certfile 0
|
||||
```
|
||||
|
||||
After executing the above command, the real-time speech transcription service will be started. If the model is specified as a ModelScope model id, the following models will be automatically downloaded from ModelScope:
|
||||
[FSMN-VAD](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary),
|
||||
[Paraformer-lagre](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary),
|
||||
[CT-Transformer](https://www.modelscope.cn/models/damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx/summary),
|
||||
[FST-ITN](https://www.modelscope.cn/models/thuduj12/fst_itn_zh/summary),
|
||||
[Ngram lm](https://www.modelscope.cn/models/damo/speech_ngram_lm_zh-cn-ai-wesp-fst/summary)
|
||||
|
||||
If you wish to deploy your fine-tuned model (e.g., 10epoch.pb), you need to manually rename the model to model.pb and replace the original model.pb in ModelScope. Then, specify the path as `model_dir`.
|
||||
|
||||
## Starting the client
|
||||
After completing the deployment of FunASR offline file transcription service on the server, you can test and use the service by following these steps. Currently, FunASR-bin supports multiple ways to start the client. The following are command-line examples based on python-client, c++-client, and custom client Websocket communication protocol:
|
||||
|
||||
### python-client
|
||||
```shell
|
||||
python funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "./data/wav.scp" --send_without_sleep --output_dir "./results"
|
||||
```
|
||||
Introduction to command parameters:
|
||||
```text
|
||||
--host: the IP address of the server. It can be set to 127.0.0.1 for local testing.
|
||||
--port: the port number of the server listener.
|
||||
--audio_in: the audio input. Input can be a path to a wav file or a wav.scp file (a Kaldi-formatted wav list in which each line includes a wav_id followed by a tab and a wav_path).
|
||||
--output_dir: the path to the recognition result output.
|
||||
--ssl: whether to use SSL encryption. The default is to use SSL.
|
||||
--mode: offline mode.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use_itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### c++-client
|
||||
```shell
|
||||
. /funasr-wss-client --server-ip 127.0.0.1 --port 10095 --wav-path test.wav --thread-num 1 --is-ssl 1
|
||||
```
|
||||
|
||||
Introduction to command parameters:
|
||||
```text
|
||||
--server-ip: the IP address of the server. It can be set to 127.0.0.1 for local testing.
|
||||
--port: the port number of the server listener.
|
||||
--wav-path: the audio input. Input can be a path to a wav file or a wav.scp file (a Kaldi-formatted wav list in which each line includes a wav_id followed by a tab and a wav_path).
|
||||
--is-ssl: whether to use SSL encryption. The default is to use SSL.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use-itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### Custom client
|
||||
If you want to define your own client, see the [Websocket communication protocol](./websocket_protocol.md)
|
||||
|
||||
## How to customize service deployment
|
||||
The code for FunASR-runtime is open source. If the server and client cannot fully meet your needs, you can further develop them based on your own requirements:
|
||||
|
||||
### C++ client
|
||||
https://github.com/modelscope/FunASR/tree/main/runtime/websocket
|
||||
|
||||
### Python client
|
||||
https://github.com/modelscope/FunASR/tree/main/runtime/python/websocket
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
# FunASR离线文件转写服务GPU版本开发指南
|
||||
|
||||
(简体中文|[English](SDK_advanced_guide_offline_gpu.md))
|
||||
|
||||
FunASR离线文件转写GPU软件包，提供了一款功能强大的语音离线文件转写服务。拥有完整的语音识别链路，结合了语音端点检测、语音识别、标点等模型，可以将几十个小时的长音频与视频识别成带标点的文字，而且支持上百路请求同时进行转写。输出为带标点的文字，含有字级别时间戳，支持ITN与用户自定义热词等。服务端集成有ffmpeg，支持各种音视频格式输入。软件包提供有html、python、c++、java与c#等多种编程语言客户端，用户可以直接使用与进一步开发。
|
||||
|
||||
本文档为FunASR离线文件转写服务GPU版本开发指南。如果您想快速体验离线文件转写服务，可参考[快速上手](#快速上手)。
|
||||
|
||||
<img src="images/offline_structure.jpg" width="900"/>
|
||||
|
||||
| 时间 | 详情 | 镜像版本 | 镜像ID |
|
||||
|------------|---------------------------------------------------|------------------------------|--------------|
|
||||
| 2024.09.26 | 修复显存泄漏 | funasr-runtime-sdk-gpu-0.2.0 | d280bf7e495b |
|
||||
| 2024.07.01 | 优化bladedisc模型兼容性问题 | funasr-runtime-sdk-gpu-0.1.1 | 8875cbf9b99e |
|
||||
| 2024.06.27 | 离线文件转写服务GPU版本1.0 发布 | funasr-runtime-sdk-gpu-0.1.0 | b86066f4d018 |
|
||||
|
||||
## 服务器配置
|
||||
|
||||
用户可以根据自己的业务需求，选择合适的服务器配置，推荐配置为：
|
||||
- 配置1: （GPU），8核vCPU，内存32G，V100，单机可以支持大约20路的请求
|
||||
|
||||
详细性能测试报告（[点击此处](./benchmark_onnx_cpp.md)）
|
||||
|
||||
云服务厂商，针对新用户，有3个月免费试用活动，申请教程（[点击此处](https://github.com/modelscope/FunASR/blob/main/runtime/docs/aliyun_server_tutorial.md)）
|
||||
|
||||
|
||||
## 快速上手
|
||||
|
||||
### docker安装
|
||||
如果您已安装docker，忽略本步骤！!
|
||||
通过下述命令在服务器上安装docker：
|
||||
```shell
|
||||
curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/install_docker.sh；
|
||||
sudo bash install_docker.sh
|
||||
```
|
||||
docker安装失败请参考 [Docker Installation](https://alibaba-damo-academy.github.io/FunASR/en/installation/docker.html)
|
||||
|
||||
### 镜像启动
|
||||
|
||||
通过下述命令拉取并启动FunASR软件包的docker镜像：
|
||||
|
||||
```shell
|
||||
sudo docker pull \
|
||||
registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-gpu-0.2.1
|
||||
mkdir -p ./funasr-runtime-resources/models
|
||||
sudo docker run --gpus=all -p 10098:10095 -it --privileged=true \
|
||||
-v $PWD/funasr-runtime-resources/models:/workspace/models \
|
||||
registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-gpu-0.2.1
|
||||
```
|
||||
|
||||
### 服务端启动
|
||||
|
||||
docker启动之后，启动 funasr-wss-server服务程序:
|
||||
```shell
|
||||
cd FunASR/runtime
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--vad-dir iic/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--model-dir iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch \
|
||||
--punc-dir iic/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--lm-dir iic/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--hotword /workspace/models/hotwords.txt > log.txt 2>&1 &
|
||||
|
||||
***服务首次启动时会导出torchscript模型，耗时较长，请耐心等待***
|
||||
# 如果您想关闭ssl，增加参数：--certfile 0
|
||||
# 默认加载时间戳模型，如果您想使用nn热词模型进行部署，请设置--model-dir为对应模型：
|
||||
# iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch（时间戳）
|
||||
# iic/speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404（nn热词）
|
||||
# 如果您想在服务端加载热词，请在宿主机文件./funasr-runtime-resources/models/hotwords.txt配置热词（docker映射地址为/workspace/models/hotwords.txt）:
|
||||
# 每行一个热词，格式(热词 权重)：阿里巴巴 20（注：热词理论上无限制，但为了兼顾性能和效果，建议热词长度不超过10，个数不超过1k，权重1~100）
|
||||
```
|
||||
如果您想定制ngram，参考文档([如何训练LM](./lm_train_tutorial.md))
|
||||
|
||||
服务端详细参数介绍可参考[服务端用法详解](#服务端用法详解)
|
||||
|
||||
### 客户端测试与使用
|
||||
|
||||
下载客户端测试工具目录samples
|
||||
```shell
|
||||
wget https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz
|
||||
```
|
||||
我们以Python语言客户端为例，进行说明，支持多种音频格式输入（.wav, .pcm, .mp3等），也支持视频输入(.mp4等)，以及多文件列表wav.scp输入，其他版本客户端请参考文档（[点击此处](#客户端用法详解)），定制服务部署请参考[如何定制服务部署](#如何定制服务部署)
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.wav"
|
||||
```
|
||||
|
||||
## 客户端用法详解
|
||||
|
||||
在服务器上完成FunASR服务部署以后，可以通过如下的步骤来测试和使用离线文件转写服务。
|
||||
目前分别支持以下几种编程语言客户端
|
||||
|
||||
- [Python](#python-client)
|
||||
- [CPP](#cpp-client)
|
||||
- [html网页版本](#Html网页版)
|
||||
- [Java](#Java-client)
|
||||
|
||||
### python-client
|
||||
若想直接运行client进行测试，可参考如下简易说明，以python版本为例：
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline \
|
||||
--audio_in "../audio/asr_example.wav" --output_dir "./results"
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
```text
|
||||
--host 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10095 部署端口号
|
||||
--mode offline表示离线文件转写
|
||||
--audio_in 需要进行转写的音频文件，支持文件路径，文件列表wav.scp
|
||||
--thread_num 设置并发发送线程数，默认为1
|
||||
--ssl 设置是否开启ssl证书校验，默认1开启，设置为0关闭
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--use_itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### cpp-client
|
||||
进入samples/cpp目录后，可以用cpp进行测试，指令如下：
|
||||
```shell
|
||||
./funasr-wss-client --server-ip 127.0.0.1 --port 10095 --wav-path ../audio/asr_example.wav
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
```text
|
||||
--server-ip 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10095 部署端口号
|
||||
--wav-path 需要进行转写的音频文件，支持文件路径
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--thread-num 设置客户端线程数
|
||||
--use-itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### Html网页版
|
||||
在浏览器中打开 html/static/index.html，即可出现如下页面，支持麦克风输入与文件上传，直接进行体验
|
||||
|
||||
<img src="images/html.png" width="900"/>
|
||||
|
||||
### Java-client
|
||||
```shell
|
||||
FunasrWsClient --host localhost --port 10095 --audio_in ./asr_example.wav --mode offline
|
||||
```
|
||||
详细可以参考文档（[点击此处](../java/readme.md)）
|
||||
|
||||
## 服务端用法详解：
|
||||
|
||||
### 启动FunASR服务
|
||||
```shell
|
||||
cd /workspace/FunASR/runtime
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--model-dir iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch \
|
||||
--vad-dir iic/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir iic/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--lm-dir iic/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--certfile ../../../ssl_key/server.crt \
|
||||
--keyfile ../../../ssl_key/server.key \
|
||||
--hotword ../../hotwords.txt > log.txt 2>&1 &
|
||||
```
|
||||
**run_server.sh命令参数介绍**
|
||||
```text
|
||||
--download-model-dir 模型下载地址，通过设置model ID从Modelscope下载模型
|
||||
--model-dir modelscope model ID 或者 本地模型路径
|
||||
--vad-dir modelscope model ID 或者 本地模型路径
|
||||
--punc-dir modelscope model ID 或者 本地模型路径
|
||||
--lm-dir modelscope model ID 或者 本地模型路径
|
||||
--itn-dir modelscope model ID 或者 本地模型路径
|
||||
--port 服务端监听的端口号，默认为 10095
|
||||
--decoder-thread-num 服务端线程池个数(支持的最大并发路数)，
|
||||
**建议每路分配1G显存，即20G显存可配置20路并发**
|
||||
--io-thread-num 服务端启动的IO线程数
|
||||
--model-thread-num 每路识别的内部线程数(控制ONNX模型的并行)，默认为 1，
|
||||
其中建议 decoder-thread-num*model-thread-num 等于总线程数
|
||||
--certfile ssl的证书文件，默认为：../../../ssl_key/server.crt，如果需要关闭ssl，参数设置为0
|
||||
--keyfile ssl的密钥文件，默认为：../../../ssl_key/server.key
|
||||
--hotword 热词文件路径，每行一个热词，格式：热词 权重(例如:阿里巴巴 20)，
|
||||
如果客户端提供热词，则与客户端提供的热词合并一起使用，服务端热词全局生效，客户端热词只针对对应客户端生效。
|
||||
```
|
||||
|
||||
### 关闭FunASR服务
|
||||
```text
|
||||
# 查看 funasr-wss-server 对应的PID
|
||||
ps -x | grep funasr-wss-server
|
||||
kill -9 PID
|
||||
```
|
||||
|
||||
### 修改模型及其他参数
|
||||
替换正在使用的模型或者其他参数，需先关闭FunASR服务，修改需要替换的参数，并重新启动FunASR服务。其中模型需为ModelScope中的ASR/VAD/PUNC模型，或者从ModelScope中模型finetune后的模型。
|
||||
```text
|
||||
# 例如替换ASR模型为 iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch，则如下设置参数 --model-dir
|
||||
--model-dir iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch
|
||||
# 设置端口号 --port
|
||||
--port <port number>
|
||||
# 设置服务端启动的推理线程数 --decoder-thread-num
|
||||
--decoder-thread-num <decoder thread num>
|
||||
# 设置服务端启动的IO线程数 --io-thread-num
|
||||
--io-thread-num <io thread num>
|
||||
# 关闭SSL证书
|
||||
--certfile 0
|
||||
```
|
||||
|
||||
执行上述指令后，启动离线文件转写服务。如果模型指定为ModelScope中model id，会自动从MoldeScope中下载如下模型：
|
||||
[FSMN-VAD模型](https://www.modelscope.cn/models/iic/speech_fsmn_vad_zh-cn-16k-common-onnx/summary),
|
||||
[Paraformer-large模型](https://www.modelscope.cn/models/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary),
|
||||
[CT-Transformer标点预测模型](https://www.modelscope.cn/models/iic/punc_ct-transformer_cn-en-common-vocab471067-large-onnx/summary),
|
||||
[基于FST的中文ITN](https://www.modelscope.cn/models/thuduj12/fst_itn_zh/summary),
|
||||
[Ngram中文语言模型](https://www.modelscope.cn/models/iic/speech_ngram_lm_zh-cn-ai-wesp-fst/summary)
|
||||
|
||||
如果，您希望部署您finetune后的模型（例如10epoch.pb），需要手动将模型重命名为model.pb，并将原modelscope中模型model.pb替换掉，将路径指定为`model_dir`即可。
|
||||
@@ -0,0 +1,286 @@
|
||||
# FunASR离线文件转写服务开发指南
|
||||
|
||||
(简体中文|[English](SDK_advanced_guide_offline.md))
|
||||
|
||||
FunASR离线文件转写软件包，提供了一款功能强大的语音离线文件转写服务。拥有完整的语音识别链路，结合了语音端点检测、语音识别、标点等模型，可以将几十个小时的长音频与视频识别成带标点的文字，而且支持上百路请求同时进行转写。输出为带标点的文字，含有字级别时间戳，支持ITN与用户自定义热词等。服务端集成有ffmpeg，支持各种音视频格式输入。软件包提供有html、python、c++、java与c#等多种编程语言客户端，用户可以直接使用与进一步开发。
|
||||
|
||||
本文档为FunASR离线文件转写服务开发指南。如果您想快速体验离线文件转写服务，可参考[快速上手](#快速上手)。
|
||||
|
||||
<img src="images/offline_structure.jpg" width="900"/>
|
||||
|
||||
| 时间 | 详情 | 镜像版本 | 镜像ID |
|
||||
|------------|---------------------------------------------------|------------------------------|--------------|
|
||||
| 2024.09.26 | 修复内存泄漏、支持SensevoiceSmall onnx模型 | funasr-runtime-sdk-cpu-0.4.6 | 8651c6b8a1ae |
|
||||
| 2024.05.15 | 适配FunASR 1.0模型结构 | funasr-runtime-sdk-cpu-0.4.5 | 058b9882ae67 |
|
||||
| 2024.03.05 | docker镜像支持arm64平台，升级modelscope版本 | funasr-runtime-sdk-cpu-0.4.4 | 2dc87b86dc49 |
|
||||
| 2024.01.25 | 优化vad数据处理方式，大幅降低峰值内存占用；内存泄漏优化| funasr-runtime-sdk-cpu-0.4.2 | befdc7b179ed |
|
||||
| 2024.01.08 | 优化句子级时间戳json格式 | funasr-runtime-sdk-cpu-0.4.1 | 0250f8ef981b |
|
||||
| 2024.01.03 | 新增支持8k模型、优化时间戳不匹配问题及增加句子级别时间戳、优化英文单词fst热词效果、支持自动化配置线程参数，同时修复已知的crash问题及内存泄漏问题 | funasr-runtime-sdk-cpu-0.4.0 | c4483ee08f04 |
|
||||
| 2023.11.08 | 支持标点大模型、支持Ngram模型、支持fst热词、支持服务端加载热词、runtime结构变化适配 | funasr-runtime-sdk-cpu-0.3.0 | caa64bddbb43 |
|
||||
| 2023.09.19 | 支持ITN模型 | funasr-runtime-sdk-cpu-0.2.2 | 2c5286be13e9 |
|
||||
| 2023.08.22 | 集成ffmpeg支持多种音视频输入、支持热词模型、支持时间戳模型 | funasr-runtime-sdk-cpu-0.2.0 | 1ad3d19e0707 |
|
||||
| 2023.07.03 | 1.0 发布 | funasr-runtime-sdk-cpu-0.1.0 | 1ad3d19e0707 |
|
||||
|
||||
## 服务器配置
|
||||
|
||||
用户可以根据自己的业务需求，选择合适的服务器配置，推荐配置为：
|
||||
- 配置1: （X86，计算型），4核vCPU，内存8G，单机可以支持大约32路的请求
|
||||
- 配置2: （X86，计算型），16核vCPU，内存32G，单机可以支持大约64路的请求
|
||||
- 配置3: （X86，计算型），64核vCPU，内存128G，单机可以支持大约200路的请求
|
||||
|
||||
详细性能测试报告（[点击此处](./benchmark_onnx_cpp.md)）
|
||||
|
||||
云服务厂商，针对新用户，有3个月免费试用活动，申请教程（[点击此处](https://github.com/modelscope/FunASR/blob/main/runtime/docs/aliyun_server_tutorial.md)）
|
||||
|
||||
|
||||
## 快速上手
|
||||
|
||||
### docker安装
|
||||
如果您已安装docker，忽略本步骤！!
|
||||
通过下述命令在服务器上安装docker：
|
||||
```shell
|
||||
curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/install_docker.sh；
|
||||
sudo bash install_docker.sh
|
||||
```
|
||||
docker安装失败请参考 [Docker Installation](https://alibaba-damo-academy.github.io/FunASR/en/installation/docker.html)
|
||||
|
||||
### 镜像启动
|
||||
|
||||
通过下述命令拉取并启动FunASR软件包的docker镜像：
|
||||
|
||||
```shell
|
||||
sudo docker pull \
|
||||
registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-cpu-0.4.7
|
||||
mkdir -p ./funasr-runtime-resources/models
|
||||
sudo docker run -p 10095:10095 -it --privileged=true \
|
||||
-v $PWD/funasr-runtime-resources/models:/workspace/models \
|
||||
registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-cpu-0.4.7
|
||||
```
|
||||
|
||||
### 服务端启动
|
||||
|
||||
docker启动之后，进入到docker里边启动 funasr-wss-server服务程序：
|
||||
```shell
|
||||
cd FunASR/runtime
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--model-dir damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--hotword /workspace/models/hotwords.txt > log.txt 2>&1 &
|
||||
|
||||
# 如果您想关闭ssl，增加参数：--certfile 0
|
||||
# 如果您想使用SenseVoiceSmall模型、时间戳、nn热词模型进行部署，请设置--model-dir为对应模型：
|
||||
# iic/SenseVoiceSmall-onnx
|
||||
# damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx（时间戳）
|
||||
# damo/speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404-onnx（nn热词）
|
||||
# 如果您想在服务端加载热词，请在宿主机文件./funasr-runtime-resources/models/hotwords.txt配置热词（docker映射地址为/workspace/models/hotwords.txt）:
|
||||
# 每行一个热词，格式(热词 权重)：阿里巴巴 20（注：热词理论上无限制，但为了兼顾性能和效果，建议热词长度不超过10，个数不超过1k，权重1~100）
|
||||
# SenseVoiceSmall-onnx识别结果中“<|zh|><|NEUTRAL|><|Speech|> ”分别为对应的语种、情感、事件信息
|
||||
```
|
||||
如果您想定制ngram，参考文档([如何训练LM](./lm_train_tutorial.md))
|
||||
|
||||
如果您想部署8k的模型，请使用如下命令启动服务：
|
||||
```shell
|
||||
cd FunASR/runtime
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-8k-common-onnx \
|
||||
--model-dir damo/speech_paraformer_asr_nat-zh-cn-8k-common-vocab8358-tensorflow1-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst-token8358 \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--hotword /workspace/models/hotwords.txt > log.txt 2>&1 &
|
||||
```
|
||||
|
||||
服务端详细参数介绍可参考[服务端用法详解](#服务端用法详解)
|
||||
|
||||
### 客户端测试与使用
|
||||
|
||||
下载客户端测试工具目录samples
|
||||
```shell
|
||||
wget https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz
|
||||
```
|
||||
我们以Python语言客户端为例，进行说明，支持多种音频格式输入（.wav, .pcm, .mp3等），也支持视频输入(.mp4等)，以及多文件列表wav.scp输入，其他版本客户端请参考文档（[点击此处](#客户端用法详解)），定制服务部署请参考[如何定制服务部署](#如何定制服务部署)
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.wav"
|
||||
```
|
||||
|
||||
## 客户端用法详解
|
||||
|
||||
在服务器上完成FunASR服务部署以后，可以通过如下的步骤来测试和使用离线文件转写服务。
|
||||
目前分别支持以下几种编程语言客户端
|
||||
|
||||
- [Python](#python-client)
|
||||
- [CPP](#cpp-client)
|
||||
- [html网页版本](#Html网页版)
|
||||
- [Java](#Java-client)
|
||||
|
||||
### python-client
|
||||
若想直接运行client进行测试，可参考如下简易说明，以python版本为例：
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline \
|
||||
--audio_in "../audio/asr_example.wav" --output_dir "./results"
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
```text
|
||||
--host 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10095 部署端口号
|
||||
--mode offline表示离线文件转写
|
||||
--audio_in 需要进行转写的音频文件，支持文件路径，文件列表wav.scp
|
||||
--thread_num 设置并发发送线程数，默认为1
|
||||
--ssl 设置是否开启ssl证书校验，默认1开启，设置为0关闭
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--use_itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### cpp-client
|
||||
进入samples/cpp目录后，可以用cpp进行测试，指令如下：
|
||||
```shell
|
||||
./funasr-wss-client --server-ip 127.0.0.1 --port 10095 --wav-path ../audio/asr_example.wav
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
```text
|
||||
--server-ip 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10095 部署端口号
|
||||
--wav-path 需要进行转写的音频文件，支持文件路径
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--thread-num 设置客户端线程数
|
||||
--use-itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### Html网页版
|
||||
在浏览器中打开 html/static/index.html，即可出现如下页面，支持麦克风输入与文件上传，直接进行体验
|
||||
|
||||
<img src="images/html.png" width="900"/>
|
||||
|
||||
### Java-client
|
||||
```shell
|
||||
FunasrWsClient --host localhost --port 10095 --audio_in ./asr_example.wav --mode offline
|
||||
```
|
||||
详细可以参考文档（[点击此处](../java/readme.md)）
|
||||
|
||||
## 服务端用法详解：
|
||||
|
||||
### 启动FunASR服务
|
||||
```shell
|
||||
cd /workspace/FunASR/runtime
|
||||
nohup bash run_server.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--certfile ../../../ssl_key/server.crt \
|
||||
--keyfile ../../../ssl_key/server.key \
|
||||
--hotword ../../hotwords.txt > log.txt 2>&1 &
|
||||
```
|
||||
**run_server.sh命令参数介绍**
|
||||
```text
|
||||
--download-model-dir 模型下载地址，通过设置model ID从Modelscope下载模型
|
||||
--model-dir modelscope model ID 或者 本地模型路径
|
||||
--vad-dir modelscope model ID 或者 本地模型路径
|
||||
--punc-dir modelscope model ID 或者 本地模型路径
|
||||
--lm-dir modelscope model ID 或者 本地模型路径
|
||||
--itn-dir modelscope model ID 或者 本地模型路径
|
||||
--port 服务端监听的端口号，默认为 10095
|
||||
--decoder-thread-num 服务端线程池个数(支持的最大并发路数)，
|
||||
脚本会根据服务器线程数自动配置decoder-thread-num、io-thread-num
|
||||
--io-thread-num 服务端启动的IO线程数
|
||||
--model-thread-num 每路识别的内部线程数(控制ONNX模型的并行)，默认为 1，
|
||||
其中建议 decoder-thread-num*model-thread-num 等于总线程数
|
||||
--certfile ssl的证书文件，默认为：../../../ssl_key/server.crt，如果需要关闭ssl，参数设置为0
|
||||
--keyfile ssl的密钥文件，默认为：../../../ssl_key/server.key
|
||||
--hotword 热词文件路径，每行一个热词，格式：热词 权重(例如:阿里巴巴 20)，
|
||||
如果客户端提供热词，则与客户端提供的热词合并一起使用，服务端热词全局生效，客户端热词只针对对应客户端生效。
|
||||
```
|
||||
|
||||
### 关闭FunASR服务
|
||||
```text
|
||||
# 查看 funasr-wss-server 对应的PID
|
||||
ps -x | grep funasr-wss-server
|
||||
kill -9 PID
|
||||
```
|
||||
|
||||
### 修改模型及其他参数
|
||||
替换正在使用的模型或者其他参数，需先关闭FunASR服务，修改需要替换的参数，并重新启动FunASR服务。其中模型需为ModelScope中的ASR/VAD/PUNC模型，或者从ModelScope中模型finetune后的模型。
|
||||
```text
|
||||
# 例如替换ASR模型为 damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx，则如下设置参数 --model-dir
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
# 设置端口号 --port
|
||||
--port <port number>
|
||||
# 设置服务端启动的推理线程数 --decoder-thread-num
|
||||
--decoder-thread-num <decoder thread num>
|
||||
# 设置服务端启动的IO线程数 --io-thread-num
|
||||
--io-thread-num <io thread num>
|
||||
# 关闭SSL证书
|
||||
--certfile 0
|
||||
```
|
||||
|
||||
执行上述指令后，启动离线文件转写服务。如果模型指定为ModelScope中model id，会自动从MoldeScope中下载如下模型：
|
||||
[FSMN-VAD模型](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary),
|
||||
[Paraformer-lagre模型](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx/summary),
|
||||
[CT-Transformer标点预测模型](https://www.modelscope.cn/models/damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx/summary),
|
||||
[基于FST的中文ITN](https://www.modelscope.cn/models/thuduj12/fst_itn_zh/summary),
|
||||
[Ngram中文语言模型](https://www.modelscope.cn/models/damo/speech_ngram_lm_zh-cn-ai-wesp-fst/summary)
|
||||
|
||||
如果，您希望部署您finetune后的模型（例如10epoch.pb），需要手动将模型重命名为model.pb，并将原modelscope中模型model.pb替换掉，将路径指定为`model_dir`即可。
|
||||
|
||||
------------------
|
||||
|
||||
## 如何定制服务部署
|
||||
|
||||
FunASR-runtime的代码已开源，如果服务端和客户端不能很好的满足您的需求，您可以根据自己的需求进行进一步的开发：
|
||||
### c++ 客户端：
|
||||
|
||||
https://github.com/modelscope/FunASR/tree/main/runtime/websocket
|
||||
|
||||
### python 客户端：
|
||||
|
||||
https://github.com/modelscope/FunASR/tree/main/runtime/python/websocket
|
||||
|
||||
### 自定义客户端：
|
||||
|
||||
如果您想定义自己的client，参考[websocket通信协议](./websocket_protocol_zh.md)
|
||||
|
||||
### c++ 服务端：
|
||||
|
||||
#### VAD
|
||||
```c++
|
||||
// VAD模型的使用分为FsmnVadInit和FsmnVadInfer两个步骤：
|
||||
FUNASR_HANDLE vad_hanlde=FsmnVadInit(model_path, thread_num);
|
||||
// 其中：model_path 包含"model-dir"、"quantize"，thread_num为onnx线程数；
|
||||
FUNASR_RESULT result=FsmnVadInfer(vad_hanlde, wav_file.c_str(), NULL, 16000);
|
||||
// 其中：vad_hanlde为FunOfflineInit返回值，wav_file为音频路径，sampling_rate为采样率(默认16k)
|
||||
```
|
||||
|
||||
使用示例详见：https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline-vad.cpp
|
||||
|
||||
#### ASR
|
||||
```text
|
||||
// ASR模型的使用分为FunOfflineInit和FunOfflineInfer两个步骤：
|
||||
FUNASR_HANDLE asr_hanlde=FunOfflineInit(model_path, thread_num);
|
||||
// 其中：model_path 包含"model-dir"、"quantize"，thread_num为onnx线程数；
|
||||
FUNASR_RESULT result=FunOfflineInfer(asr_hanlde, wav_file.c_str(), RASR_NONE, NULL, 16000);
|
||||
// 其中：asr_hanlde为FunOfflineInit返回值，wav_file为音频路径，sampling_rate为采样率(默认16k)
|
||||
```
|
||||
|
||||
使用示例详见：https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline.cpp
|
||||
|
||||
#### PUNC
|
||||
```text
|
||||
// PUNC模型的使用分为CTTransformerInit和CTTransformerInfer两个步骤：
|
||||
FUNASR_HANDLE punc_hanlde=CTTransformerInit(model_path, thread_num);
|
||||
// 其中：model_path 包含"model-dir"、"quantize"，thread_num为onnx线程数；
|
||||
FUNASR_RESULT result=CTTransformerInfer(punc_hanlde, txt_str.c_str(), RASR_NONE, NULL);
|
||||
// 其中：punc_hanlde为CTTransformerInit返回值，txt_str为文本
|
||||
```
|
||||
使用示例详见：https://github.com/modelscope/FunASR/blob/main/runtime/onnxruntime/bin/funasr-onnx-offline-punc.cpp
|
||||
@@ -0,0 +1,157 @@
|
||||
# Real-time Speech Transcription Service Development Guide
|
||||
([简体中文](SDK_advanced_guide_online_zh.md)|English)
|
||||
|
||||
[//]: # (FunASR provides a real-time speech transcription service that can be easily deployed on local or cloud servers, with the FunASR runtime-SDK as the core. It integrates the speech endpoint detection (VAD), Paraformer-large non-streaming speech recognition (ASR), Paraformer-large streaming speech recognition (ASR), punctuation (PUNC), and other related capabilities open-sourced by the speech laboratory of DAMO Academy on the Modelscope community. The software package can perform real-time speech-to-text transcription, and can also accurately transcribe text at the end of sentences for high-precision output. The output text contains punctuation and supports high-concurrency multi-channel requests.)
|
||||
FunASR Real-time Speech Recognition Software Package integrates real-time versions of speech endpoint detection model, speech recognition model, punctuation prediction model, and so on. By using multiple models collaboratively, it can perform real-time speech-to-text conversion, as well as high-precision transcription correction at the end of a sentence, with punctuation included in the output text. It supports multiple concurrent requests. Depending on the user's scenarios, it supports three service modes: real-time speech recognition service (online), non-real-time single-sentence transcription (offline), and real-time and non-real-time integrated collaboration (2pass). The software package provides client libraries in various programming languages such as HTML, Python, C++, Java, and C#, allowing users to use and further develop the software.
|
||||
|
||||
<img src="images/online_structure.png" width="900"/>
|
||||
|
||||
| TIME | INFO | IMAGE VERSION | IMAGE ID |
|
||||
|------------|-------------------------------------------------------------------------------------|-------------------------------------|--------------|
|
||||
| 2024.10.29 | The 2pass-offline mode supports the SensevoiceSmal model | funasr-runtime-sdk-online-cpu-0.1.12 | f5febc5cf13a |
|
||||
| 2024.09.26 | Fix memory leak | funasr-runtime-sdk-online-cpu-0.1.11 | e51a36c42771 |
|
||||
| 2024.05.15 | Adapting to FunASR 1.0 model structure | funasr-runtime-sdk-online-cpu-0.1.10 | 1c2adfcff84d |
|
||||
| 2024.03.05 | docker image supports ARM64 platform, update modelscope | funasr-runtime-sdk-online-cpu-0.1.9 | 4a875e08c7a2 |
|
||||
| 2024.01.25 | Optimization of the client-side | funasr-runtime-sdk-online-cpu-0.1.7 | 2aa23805572e |
|
||||
| 2024.01.03 | The 2pass-offline mode supports Ngram language model decoding and WFST hotwords, while also addressing known crash issues and memory leak problems | funasr-runtime-sdk-online-cpu-0.1.6 | f99925110d27 |
|
||||
| 2023.11.09 | fix bug: without online results | funasr-runtime-sdk-online-cpu-0.1.5 | b16584b6d38b |
|
||||
| 2023.11.08 | supporting server-side loading of hotwords, adaptation to runtime structure changes | funasr-runtime-sdk-online-cpu-0.1.4 | 691974017c38 |
|
||||
| 2023.09.19 | supporting hotwords, timestamps, and ITN model in 2pass mode | funasr-runtime-sdk-online-cpu-0.1.2 | 7222c5319bcf |
|
||||
| 2023.08.11 | addressing some known bugs (including server crashes) | funasr-runtime-sdk-online-cpu-0.1.1 | bdbdd0b27dee |
|
||||
| 2023.08.07 | 1.0 released | funasr-runtime-sdk-online-cpu-0.1.0 | bdbdd0b27dee |
|
||||
|
||||
## Quick Start
|
||||
### Docker install
|
||||
If you have already installed Docker, ignore this step!
|
||||
```shell
|
||||
curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/install_docker.sh;
|
||||
sudo bash install_docker.sh
|
||||
```
|
||||
If you do not have Docker installed, please refer to [Docker Installation](https://alibaba-damo-academy.github.io/FunASR/en/installation/docker.html)
|
||||
|
||||
### Pull Docker Image
|
||||
Use the following command to pull and start the FunASR software package docker image:
|
||||
```shell
|
||||
sudo docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.13
|
||||
mkdir -p ./funasr-runtime-resources/models
|
||||
sudo docker run -p 10096:10095 -it --privileged=true -v $PWD/funasr-runtime-resources/models:/workspace/models registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.13
|
||||
```
|
||||
|
||||
### Launching the Server
|
||||
|
||||
After Docker is launched, start the funasr-wss-server-2pass service program:
|
||||
```shell
|
||||
cd FunASR/runtime
|
||||
nohup bash run_server_2pass.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx \
|
||||
--online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx \
|
||||
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--itn-dir thuduj12/fst_itn_zh > log.txt 2>&1 &
|
||||
|
||||
# If you want to close ssl，please add：--certfile 0
|
||||
```
|
||||
For a more detailed description of server parameters, please refer to Server Introduction
|
||||
### Client Testing and Usage
|
||||
|
||||
Download the client testing tool directory `samples`:
|
||||
```shell
|
||||
wget https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz
|
||||
```
|
||||
For illustration, we will use the Python language client, which supports audio formats (.wav, .pcm) and a multi-file list wav.scp input.
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10096 --mode 2pass
|
||||
```
|
||||
|
||||
------------------
|
||||
|
||||
## Client Usage Details
|
||||
|
||||
After completing the FunASR service deployment on the server, you can test and use the offline file transcription service by following these steps. Currently, the following programming language client versions are supported:
|
||||
|
||||
- [Python](./SDK_tutorial_online.md#python-client)
|
||||
- [CPP](./SDK_tutorial_online.md#cpp-client)
|
||||
- [Html](./SDK_tutorial_online.md#html-client)
|
||||
- [Java](./SDK_tutorial_online.md#java-client)
|
||||
- [C\#](./SDK_tutorial_online.md#c\#)
|
||||
|
||||
For more detailed usage, please click on the links above. For more client version support, please refer to [WebSocket/GRPC Protocol](./websocket_protocol_zh.md).
|
||||
|
||||
|
||||
## Server Introduction
|
||||
|
||||
Use the flollowing script to start the server ：
|
||||
```shell
|
||||
cd /workspace/FunASR/runtime
|
||||
nohup bash run_server_2pass.sh \
|
||||
--model-dir damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx \
|
||||
--online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx \
|
||||
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--certfile ../../../ssl_key/server.crt \
|
||||
--keyfile ../../../ssl_key/server.key \
|
||||
--hotword ../../hotwords.txt > log.txt 2>&1 &
|
||||
|
||||
# If you want to close ssl，please add：--certfile 0
|
||||
# If you want to deploy the SenseVoiceSmall, timestamp or nn hotword model, please set --model-dir to the corresponding model:
|
||||
# iic/SenseVoiceSmall-onnx
|
||||
# speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx（timestamp）
|
||||
# damo/speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404-onnx（hotword）
|
||||
# The ASR results “<|zh|><|NEUTRAL|><|Speech|> ” in SenseVoiceSmall-onnx means spoken language、speech emotion、audio event
|
||||
```
|
||||
|
||||
### More details about the script run_server_2pass.sh:
|
||||
```text
|
||||
--download-model-dir: Model download address, download models from Modelscope by setting the model ID.
|
||||
--model-dir: modelscope model ID or local model path.
|
||||
--online-model-dir modelscope model ID
|
||||
--vad-dir: modelscope model ID or local model path.
|
||||
--punc-dir: modelscope model ID or local model path.
|
||||
--lm-dir modelscope model ID or local model path.
|
||||
--itn-dir modelscope model ID or local model path.
|
||||
--port: Port number that the server listens on. Default is 10095.
|
||||
--decoder-thread-num: The number of thread pools on the server side that can handle concurrent requests.
|
||||
The script will automatically configure parameters decoder-thread-num and io-thread-num based on the server's thread count.
|
||||
--io-thread-num: Number of IO threads that the server starts.
|
||||
--model-thread-num: The number of internal threads for each recognition route to control the parallelism of the ONNX model.
|
||||
The default value is 1. It is recommended that decoder-thread-num * model-thread-num equals the total number of threads.
|
||||
--certfile <string>: SSL certificate file. Default is ../../../ssl_key/server.crt. If you want to close ssl，set 0
|
||||
--keyfile <string>: SSL key file. Default is ../../../ssl_key/server.key.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20), if the client provides hot words, then combined with the hot words provided by the client.
|
||||
```
|
||||
|
||||
### Shutting Down the FunASR Service
|
||||
```text
|
||||
# Check the PID of the funasr-wss-server-2pass process
|
||||
ps -x | grep funasr-wss-server-2pass
|
||||
kill -9 PID
|
||||
```
|
||||
|
||||
### Modifying Models and Other Parameters
|
||||
To replace the currently used model or other parameters, you need to first shut down the FunASR service, make the necessary modifications to the parameters you want to replace, and then restart the FunASR service. The model should be either an ASR/VAD/PUNC model from ModelScope or a fine-tuned model obtained from ModelScope.
|
||||
```text
|
||||
# For example, to replace the ASR model with damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx, use the following parameter setting --model-dir
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
# Set the port number using --port
|
||||
--port <port number>
|
||||
# Set the number of inference threads the server will start using --decoder-thread-num
|
||||
--decoder-thread-num <decoder thread num>
|
||||
# Set the number of IO threads the server will start using --io-thread-num
|
||||
--io-thread-num <io thread num>
|
||||
# Disable SSL certificate
|
||||
--certfile 0
|
||||
```
|
||||
|
||||
After executing the above command, the real-time speech transcription service will be started. If the model is specified as a ModelScope model id, the following models will be automatically downloaded from ModelScope:
|
||||
[FSMN-VAD model](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary),
|
||||
[Paraformer-lagre online](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx/summary),
|
||||
[Paraformer-lagre](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx/summary),
|
||||
[CT-Transformer](https://www.modelscope.cn/models/damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx/summary),
|
||||
[FST-ITN](https://www.modelscope.cn/models/thuduj12/fst_itn_zh/summary)
|
||||
|
||||
If you wish to deploy your fine-tuned model (e.g., 10epoch.pb), you need to manually rename the model to model.pb and replace the original model.pb in ModelScope. Then, specify the path as `model_dir`.
|
||||
@@ -0,0 +1,169 @@
|
||||
# FunASR实时语音听写服务开发指南
|
||||
(简体中文|[English](SDK_advanced_guide_online.md))
|
||||
|
||||
[//]: # (FunASR提供可便捷本地或者云端服务器部署的实时语音听写服务，内核为FunASR已开源runtime-SDK。)
|
||||
[//]: # (集成了达摩院语音实验室在Modelscope社区开源的语音端点检测(VAD)、Paraformer-large非流式语音识别(ASR)、Paraformer-large流式语音识别(ASR)、标点(PUNC) 等相关能力。软件包既可以实时地进行语音转文字，而且能够在说话句尾用高精度的转写文字修正输出，输出文字带有标点，支持高并发多路请求)
|
||||
FunASR实时语音听写软件包，集成了实时版本的语音端点检测模型、语音识别、标点预测模型等。采用多模型协同，既可以实时的进行语音转文字，也可以在说话句尾用高精度转写文字修正输出，输出文字带有标点，支持多路请求。依据使用者场景不同，支持实时语音听写服务（online）、非实时一句话转写（offline）与实时与非实时一体化协同（2pass）3种服务模式。软件包提供有html、python、c++、java与c#等多种编程语言客户端，用户可以直接使用与进一步开发。
|
||||
|
||||
|
||||
本文档为FunASR实时转写服务开发指南。如果您想快速体验实时语音听写服务，可参考[快速上手](#快速上手)。
|
||||
|
||||
<img src="images/online_structure.png" width="900"/>
|
||||
|
||||
| 时间 | 详情 | 镜像版本 | 镜像ID |
|
||||
|:-----------|:----------------------------------|--------------------------------------|--------------|
|
||||
| 2024.10.29 | 2pass-offline模式支持SensevoiceSmall模型 | funasr-runtime-sdk-online-cpu-0.1.12 | f5febc5cf13a |
|
||||
| 2024.09.26 | 修复内存泄漏 | funasr-runtime-sdk-online-cpu-0.1.11 | e51a36c42771 |
|
||||
| 2024.05.15 | 适配FunASR 1.0模型结构 | funasr-runtime-sdk-online-cpu-0.1.10 | 1c2adfcff84d |
|
||||
| 2024.03.05 | docker镜像支持arm64平台，升级modelscope版本 | funasr-runtime-sdk-online-cpu-0.1.9 | 4a875e08c7a2 |
|
||||
| 2024.01.25 | 客户端优化| funasr-runtime-sdk-online-cpu-0.1.7 | 2aa23805572e |
|
||||
| 2024.01.03 | 2pass-offline模式支持Ngram语言模型解码、wfst热词，同时修复已知的crash问题及内存泄漏问题 | funasr-runtime-sdk-online-cpu-0.1.6 | f99925110d27 |
|
||||
| 2023.11.09 | 修复无实时结果问题 | funasr-runtime-sdk-online-cpu-0.1.5 | b16584b6d38b |
|
||||
| 2023.11.08 | 支持服务端加载热词(更新热词通信协议)、runtime结构变化适配 | funasr-runtime-sdk-online-cpu-0.1.4 | 691974017c38 |
|
||||
| 2023.09.19 | 2pass模式支持热词、时间戳、ITN模型 | funasr-runtime-sdk-online-cpu-0.1.2 | 7222c5319bcf |
|
||||
| 2023.08.11 | 修复了部分已知的bug(包括server崩溃等) | funasr-runtime-sdk-online-cpu-0.1.1 | bdbdd0b27dee |
|
||||
| 2023.08.07 | 1.0 发布 | funasr-runtime-sdk-online-cpu-0.1.0 | bdbdd0b27dee |
|
||||
|
||||
|
||||
## 快速上手
|
||||
|
||||
### docker安装
|
||||
如果您已安装docker，忽略本步骤！!
|
||||
通过下述命令在服务器上安装docker：
|
||||
```shell
|
||||
curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/install_docker.sh
|
||||
sudo bash install_docker.sh
|
||||
```
|
||||
docker安装失败请参考 [Docker Installation](https://alibaba-damo-academy.github.io/FunASR/en/installation/docker.html)
|
||||
|
||||
### 镜像启动
|
||||
通过下述命令拉取并启动FunASR软件包的docker镜像：
|
||||
|
||||
```shell
|
||||
sudo docker pull \
|
||||
registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.13
|
||||
mkdir -p ./funasr-runtime-resources/models
|
||||
sudo docker run -p 10096:10095 -it --privileged=true \
|
||||
-v $PWD/funasr-runtime-resources/models:/workspace/models \
|
||||
registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.13
|
||||
```
|
||||
|
||||
### 服务端启动
|
||||
|
||||
docker启动之后，启动 funasr-wss-server-2pass服务程序：
|
||||
```shell
|
||||
cd FunASR/runtime
|
||||
nohup bash run_server_2pass.sh \
|
||||
--download-model-dir /workspace/models \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--model-dir damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx \
|
||||
--online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx \
|
||||
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--hotword /workspace/models/hotwords.txt > log.txt 2>&1 &
|
||||
|
||||
# 如果您想关闭ssl，增加参数：--certfile 0
|
||||
# 如果您想使用SenseVoiceSmall模型、时间戳、nn热词模型进行部署，请设置--model-dir为对应模型：
|
||||
# iic/SenseVoiceSmall-onnx
|
||||
# damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx（时间戳）
|
||||
# damo/speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404-onnx（nn热词）
|
||||
# 如果您想在服务端加载热词，请在宿主机文件./funasr-runtime-resources/models/hotwords.txt配置热词（docker映射地址为/workspace/models/hotwords.txt）:
|
||||
# 每行一个热词，格式(热词 权重)：阿里巴巴 20（注：热词理论上无限制，但为了兼顾性能和效果，建议热词长度不超过10，个数不超过1k，权重1~100）
|
||||
# SenseVoiceSmall-onnx识别结果中“<|zh|><|NEUTRAL|><|Speech|> ”分别为对应的语种、情感、事件信息
|
||||
```
|
||||
服务端详细参数介绍可参考[服务端用法详解](#服务端用法详解)
|
||||
### 客户端测试与使用
|
||||
|
||||
下载客户端测试工具目录samples
|
||||
```shell
|
||||
wget https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz
|
||||
```
|
||||
我们以Python语言客户端为例，进行说明，支持音频格式（.wav, .pcm），以及多文件列表wav.scp输入，其他版本客户端请参考文档（[点击此处](#客户端用法详解)）。
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10096 --mode 2pass
|
||||
```
|
||||
|
||||
------------------
|
||||
|
||||
## 客户端用法详解
|
||||
|
||||
在服务器上完成FunASR服务部署以后，可以通过如下的步骤来测试和使用离线文件转写服务。
|
||||
目前分别支持以下几种编程语言客户端
|
||||
|
||||
- [Python](./SDK_tutorial_online_zh.md#python-client)
|
||||
- [CPP](./SDK_tutorial_online_zh.md#cpp-client)
|
||||
- [html网页版本](./SDK_tutorial_online_zh.md#html-client)
|
||||
- [Java](./SDK_tutorial_online_zh.md#java-client)
|
||||
- [c\#](./SDK_tutorial_online_zh.md#c\#)
|
||||
|
||||
详细用法可以点击进入查看。更多版本客户端支持请参考[websocket/grpc协议](./websocket_protocol_zh.md)
|
||||
|
||||
## 服务端用法详解：
|
||||
|
||||
### 启动FunASR服务
|
||||
```shell
|
||||
cd /workspace/FunASR/runtime
|
||||
nohup bash run_server_2pass.sh \
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx \
|
||||
--online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx \
|
||||
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx \
|
||||
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--itn-dir thuduj12/fst_itn_zh \
|
||||
--certfile ../../../ssl_key/server.crt \
|
||||
--keyfile ../../../ssl_key/server.key \
|
||||
--hotword ../../hotwords.txt > log.txt 2>&1 &
|
||||
```
|
||||
**run_server_2pass.sh命令参数介绍**
|
||||
```text
|
||||
--download-model-dir 模型下载地址，通过设置model ID从Modelscope下载模型
|
||||
--model-dir modelscope model ID 或者 本地模型路径
|
||||
--online-model-dir modelscope model ID 或者 本地模型路径
|
||||
--vad-dir modelscope model ID 或者 本地模型路径
|
||||
--punc-dir modelscope model ID 或者 本地模型路径
|
||||
--lm-dir modelscope model ID 或者 本地模型路径
|
||||
--itn-dir modelscope model ID 或者 本地模型路径
|
||||
--port 服务端监听的端口号，默认为 10095
|
||||
--decoder-thread-num 服务端线程池个数(支持的最大并发路数)，
|
||||
脚本会根据服务器线程数自动配置decoder-thread-num、io-thread-num
|
||||
--io-thread-num 服务端启动的IO线程数
|
||||
--model-thread-num 每路识别的内部线程数(控制ONNX模型的并行)，默认为 1，
|
||||
其中建议 decoder-thread-num*model-thread-num 等于总线程数
|
||||
--certfile ssl的证书文件，默认为：../../../ssl_key/server.crt，如果需要关闭ssl，参数设置为0
|
||||
--keyfile ssl的密钥文件，默认为：../../../ssl_key/server.key
|
||||
--hotword 热词文件路径，每行一个热词，格式：热词 权重(例如:阿里巴巴 20)，
|
||||
如果客户端提供热词，则与客户端提供的热词合并一起使用，服务端热词全局生效，客户端热词只针对对应客户端生效。
|
||||
```
|
||||
|
||||
### 关闭FunASR服务
|
||||
```text
|
||||
# 查看 funasr-wss-server-2pass 对应的PID
|
||||
ps -x | grep funasr-wss-server-2pass
|
||||
kill -9 PID
|
||||
```
|
||||
|
||||
### 修改模型及其他参数
|
||||
替换正在使用的模型或者其他参数，需先关闭FunASR服务，修改需要替换的参数，并重新启动FunASR服务。其中模型需为ModelScope中的ASR/VAD/PUNC模型，或者从ModelScope中模型finetune后的模型。
|
||||
```text
|
||||
# 例如替换ASR模型为 damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx，则如下设置参数 --model-dir
|
||||
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
# 设置端口号 --port
|
||||
--port <port number>
|
||||
# 设置服务端启动的推理线程数 --decoder-thread-num
|
||||
--decoder-thread-num <decoder thread num>
|
||||
# 设置服务端启动的IO线程数 --io-thread-num
|
||||
--io-thread-num <io thread num>
|
||||
# 关闭SSL证书
|
||||
--certfile 0
|
||||
```
|
||||
|
||||
执行上述指令后，启动实时语音听写服务。如果模型指定为ModelScope中model id，会自动从MoldeScope中下载如下模型：
|
||||
[FSMN-VAD模型](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary),
|
||||
[Paraformer-lagre实时模型](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx/summary ),
|
||||
[Paraformer-lagre非实时模型](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx/summary),
|
||||
[CT-Transformer标点预测模型](https://www.modelscope.cn/models/damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx/summary),
|
||||
[基于FST的中文ITN](https://www.modelscope.cn/models/thuduj12/fst_itn_zh/summary)
|
||||
|
||||
如果，您希望部署您finetune后的模型（例如10epoch.pb），需要手动将模型重命名为model.pb，并将原modelscope中模型model.pb替换掉，将路径指定为`model_dir`即可。
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
([简体中文](./SDK_tutorial_zh.md)|English)
|
||||
|
||||
# Highlights
|
||||
**FunASR offline file transcription service has been upgraded to version 2.0, which integrates ffmpeg to support various audio and video inputs. It also supports custom hotword models and timestamp models. Feel free to deploy and experience it!**
|
||||
|
||||
# FunASR Offline File Transcription Service
|
||||
|
||||
FunASR provides an offline file transcription service that can be easily deployed on a local or cloud server. The core is the FunASR open-source runtime-SDK. It integrates various capabilities such as speech endpoint detection (VAD) and Paraformer-large speech recognition (ASR) and punctuation restoration (PUNC) released by the speech laboratory of the Damo Academy in the Modelscope community. It has a complete speech recognition chain and can recognize audio or video of tens of hours into punctuated text. Moreover, it supports transcription for hundreds of simultaneous requests.
|
||||
|
||||
## Server Configuration
|
||||
|
||||
Users can choose appropriate server configurations based on their business needs. The recommended configurations are:
|
||||
- Configuration 1: (X86, computing-type) 4-core vCPU, 8GB memory, and a single machine can support about 32 requests.
|
||||
- Configuration 2: (X86, computing-type) 16-core vCPU, 32GB memory, and a single machine can support about 64 requests.
|
||||
- Configuration 3: (X86, computing-type) 64-core vCPU, 128GB memory, and a single machine can support about 200 requests.
|
||||
|
||||
Detailed performance [report](./benchmark_onnx_cpp.md)
|
||||
|
||||
Cloud service providers offer a 3-month free trial for new users. Application tutorial ([docs](./aliyun_server_tutorial.md)).
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Server Startup
|
||||
|
||||
`Note`: The one-click deployment tool process includes installing Docker, downloading Docker images, and starting the service. If the user wants to start from the FunASR Docker image, please refer to the development guide ([docs](./SDK_advanced_guide_offline.md).
|
||||
|
||||
Download the deployment tool `funasr-runtime-deploy-offline-cpu-zh.sh`
|
||||
|
||||
```shell
|
||||
curl -O https://raw.githubusercontent.com/alibaba-damo-academy/FunASR/main/runtime/deploy_tools/funasr-runtime-deploy-offline-cpu-zh.sh;
|
||||
# If there is a network problem, users in mainland China can use the following command:
|
||||
# curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/funasr-runtime-deploy-offline-cpu-zh.sh;
|
||||
```
|
||||
|
||||
Execute the deployment tool and press the Enter key at the prompt to complete the installation and deployment of the server. Currently, the convenient deployment tool only supports Linux environments. For other environments, please refer to the development guide ([docs](./SDK_advanced_guide_offline.md)).
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh install --workspace ./funasr-runtime-resources
|
||||
```
|
||||
Note: If you need to deploy the timestamp model or hotword model, select the corresponding model in step 2 of the installation and deployment process, where 1 is the paraformer-large model, 2 is the paraformer-large timestamp model, and 3 is the paraformer-large hotword model；The server loads the hotword file from: ./funasr-runtime-resources/hotowrds.txt (one hotword per line, formatted as word weight: 阿里巴巴 20).
|
||||
|
||||
### Client Testing and Usage
|
||||
|
||||
After running the above installation instructions, the client testing tool directory samples will be downloaded in the default installation directory /root/funasr-runtime-resources ([download click](https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz)).
|
||||
We take the Python language client as an example to explain that it supports multiple audio format inputs (such as .wav, .pcm, .mp3, etc.), video inputs (.mp4, etc.), and multiple file list wav.scp inputs. For other client versions, please refer to the [documentation](#Detailed-Description-of-Client-Usage).
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.wav"
|
||||
```
|
||||
|
||||
## Detailed Description of Client Usage
|
||||
|
||||
After completing the FunASR runtime-SDK service deployment on the server, you can test and use the offline file transcription service through the following steps. Currently, the following programming language client versions are supported:
|
||||
|
||||
- [Python](#python-client)
|
||||
- [CPP](#cpp-client)
|
||||
- [html](#html-client)
|
||||
- [java](#java-client)
|
||||
|
||||
For more client version support, please refer to the [development guide](./SDK_advanced_guide_offline_zh.md).
|
||||
|
||||
### python-client
|
||||
If you want to run the client directly for testing, you can refer to the following simple instructions, using the Python version as an example:
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.wav"
|
||||
```
|
||||
|
||||
Command parameter instructions:
|
||||
```text
|
||||
--host is the IP address of the FunASR runtime-SDK service deployment machine, which defaults to the local IP address (127.0.0.1). If the client and the service are not on the same server, it needs to be changed to the deployment machine IP address.
|
||||
--port 10095 deployment port number
|
||||
--mode offline represents offline file transcription
|
||||
--audio_in is the audio file that needs to be transcribed, supporting file paths and file list wav.scp
|
||||
--thread_num sets the number of concurrent sending threads, default is 1
|
||||
--ssl sets whether to enable SSL certificate verification, default is 1 to enable, and 0 to disable
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use_itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### cpp-client
|
||||
|
||||
After entering the samples/cpp directory, you can test it with CPP. The command is as follows:
|
||||
```shell
|
||||
./funasr-wss-client --server-ip 127.0.0.1 --port 10095 --wav-path ../audio/asr_example.wav
|
||||
```
|
||||
|
||||
Command parameter description:
|
||||
```text
|
||||
--server-ip specifies the IP address of the machine where the FunASR runtime-SDK service is deployed. The default value is the local IP address (127.0.0.1). If the client and the service are not on the same server, the IP address needs to be changed to the IP address of the deployment machine.
|
||||
--port specifies the deployment port number as 10095.
|
||||
--wav-path specifies the audio file to be transcribed, and supports file paths.
|
||||
--thread_num sets the number of concurrent send threads, with a default value of 1.
|
||||
--ssl sets whether to enable SSL certificate verification, with a default value of 1 for enabling and 0 for disabling.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use-itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### html-client
|
||||
|
||||
To experience it directly, open `html/static/index.html` in your browser. You will see the following page, which supports microphone input and file upload.
|
||||
<img src="images/html.png" width="900"/>
|
||||
|
||||
### java-client
|
||||
|
||||
```shell
|
||||
FunasrWsClient --host localhost --port 10095 --audio_in ./asr_example.wav --mode offline
|
||||
```
|
||||
For more details, please refer to the [docs](../java/readme.md)
|
||||
|
||||
## Server Usage Details
|
||||
|
||||
### Start the deployed FunASR service
|
||||
|
||||
If you have restarted the computer or shut down Docker after one-click deployment, you can start the FunASR service directly with the following command. The startup configuration is the same as the last one-click deployment.
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh start
|
||||
```
|
||||
|
||||
### Stop the FunASR service
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh stop
|
||||
```
|
||||
|
||||
### Release the FunASR service
|
||||
|
||||
Release the deployed FunASR service.
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh remove
|
||||
```
|
||||
|
||||
### Restart the FunASR service
|
||||
|
||||
Restart the FunASR service with the same configuration as the last one-click deployment.
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh restart
|
||||
```
|
||||
|
||||
### Replace the model and restart the FunASR service
|
||||
|
||||
Replace the currently used model, and restart the FunASR service. The model must be an ASR/VAD/PUNC model in ModelScope, or a finetuned model from ModelScope.
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update [--asr_model | --vad_model | --punc_model] <model_id or local model path>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update --asr_model damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch
|
||||
```
|
||||
|
||||
### Update parameters and restart the FunASR service
|
||||
|
||||
Update the configured parameters and restart the FunASR service to take effect. The parameters that can be updated include the host and Docker port numbers, as well as the number of inference and IO threads.
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update [--host_port | --docker_port] <port number>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update [--decode_thread_num | --io_thread_num] <the number of threads>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update [--workspace] <workspace in local>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update [--ssl] <0: close SSL; 1: open SSL, default:1>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update --decode_thread_num 32
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update --workspace /root/funasr-runtime-resources
|
||||
```
|
||||
|
||||
### Set SSL
|
||||
|
||||
SSL verification is enabled by default. If you need to disable it, you can set it when starting.
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update --ssl 0
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Contact Us
|
||||
|
||||
If you encounter any problems during use, please join our user group for feedback.
|
||||
|
||||
|
||||
| DingDing Group | Wechat |
|
||||
|:----------------------------------------------------------------------------:|:--------------------------------------------------------------:|
|
||||
| <div align="left"><img src="../../../docs/images/dingding.jpg" width="250"/> | <img src="../../../docs/images/wechat.png" width="232"/></div> |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
([简体中文](./SDK_tutorial_en_zh.md)|English)
|
||||
|
||||
# Highlights
|
||||
**FunASR offline file transcription service 1.0 has been released. Feel free to deploy and experience it!**
|
||||
|
||||
# FunASR Offline File Transcription Service
|
||||
|
||||
FunASR provides an offline file transcription service that can be easily deployed on a local or cloud server. The core is the FunASR open-source runtime-SDK. It integrates various capabilities such as speech endpoint detection (VAD) and Paraformer-large speech recognition (ASR) and punctuation restoration (PUNC) released by the speech laboratory of the Damo Academy in the Modelscope community. It has a complete speech recognition chain and can recognize audio or video of tens of hours into punctuated text. Moreover, it supports transcription for hundreds of simultaneous requests.
|
||||
|
||||
## Server Configuration
|
||||
|
||||
Users can choose appropriate server configurations based on their business needs. The recommended configurations are:
|
||||
- Configuration 1: (X86, computing-type) 4-core vCPU, 8GB memory, and a single machine can support about 32 requests.
|
||||
- Configuration 2: (X86, computing-type) 16-core vCPU, 32GB memory, and a single machine can support about 64 requests.
|
||||
- Configuration 3: (X86, computing-type) 64-core vCPU, 128GB memory, and a single machine can support about 200 requests.
|
||||
|
||||
Detailed performance [report](./benchmark_onnx_cpp.md)
|
||||
|
||||
Cloud service providers offer a 3-month free trial for new users. Application tutorial ([docs](./aliyun_server_tutorial.md)).
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Server Startup
|
||||
|
||||
`Note`: The one-click deployment tool process includes installing Docker, downloading Docker images, and starting the service. If the user wants to start from the FunASR Docker image, please refer to the development guide ([docs](./SDK_advanced_guide_offline.md).
|
||||
|
||||
Download the deployment tool `funasr-runtime-deploy-offline-cpu-en.sh`
|
||||
|
||||
```shell
|
||||
curl -O https://raw.githubusercontent.com/alibaba-damo-academy/FunASR/main/runtime/deploy_tools/funasr-runtime-deploy-offline-cpu-en.sh;
|
||||
# If there is a network problem, users in mainland China can use the following command:
|
||||
# curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/funasr-runtime-deploy-offline-cpu-en.sh;
|
||||
```
|
||||
|
||||
Execute the deployment tool and press the Enter key at the prompt to complete the installation and deployment of the server. Currently, the convenient deployment tool only supports Linux environments. For other environments, please refer to the development guide ([docs](./SDK_advanced_guide_offline_en.md)).
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh install --workspace /root/funasr-runtime-resources
|
||||
```
|
||||
|
||||
### Client Testing and Usage
|
||||
|
||||
After running the above installation instructions, the client testing tool directory samples will be downloaded in the default installation directory /root/funasr-runtime-resources ([download click](https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz)).
|
||||
We take the Python language client as an example to explain that it supports multiple audio format inputs (such as .wav, .pcm, .mp3, etc.), video inputs (.mp4, etc.), and multiple file list wav.scp inputs. For other client versions, please refer to the [documentation](#Detailed-Description-of-Client-Usage).
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.wav"
|
||||
```
|
||||
|
||||
## Detailed Description of Client Usage
|
||||
|
||||
After completing the FunASR runtime-SDK service deployment on the server, you can test and use the offline file transcription service through the following steps. Currently, the following programming language client versions are supported:
|
||||
|
||||
- [Python](#python-client)
|
||||
- [CPP](#cpp-client)
|
||||
- [html](#html-client)
|
||||
- [java](#java-client)
|
||||
|
||||
For more client version support, please refer to the [development guide](./SDK_advanced_guide_offline_zh.md).
|
||||
|
||||
### python-client
|
||||
If you want to run the client directly for testing, you can refer to the following simple instructions, using the Python version as an example:
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.wav"
|
||||
```
|
||||
|
||||
Command parameter instructions:
|
||||
```text
|
||||
--host is the IP address of the FunASR runtime-SDK service deployment machine, which defaults to the local IP address (127.0.0.1). If the client and the service are not on the same server, it needs to be changed to the deployment machine IP address.
|
||||
--port 10095 deployment port number
|
||||
--mode offline represents offline file transcription
|
||||
--audio_in is the audio file that needs to be transcribed, supporting file paths and file list wav.scp
|
||||
--thread_num sets the number of concurrent sending threads, default is 1
|
||||
--ssl sets whether to enable SSL certificate verification, default is 1 to enable, and 0 to disable
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use_itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### cpp-client
|
||||
|
||||
After entering the samples/cpp directory, you can test it with CPP. The command is as follows:
|
||||
```shell
|
||||
./funasr-wss-client --server-ip 127.0.0.1 --port 10095 --wav-path ../audio/asr_example.wav
|
||||
```
|
||||
|
||||
Command parameter description:
|
||||
```text
|
||||
--server-ip specifies the IP address of the machine where the FunASR runtime-SDK service is deployed. The default value is the local IP address (127.0.0.1). If the client and the service are not on the same server, the IP address needs to be changed to the IP address of the deployment machine.
|
||||
--port specifies the deployment port number as 10095.
|
||||
--wav-path specifies the audio file to be transcribed, and supports file paths.
|
||||
--thread_num sets the number of concurrent send threads, with a default value of 1.
|
||||
--ssl sets whether to enable SSL certificate verification, with a default value of 1 for enabling and 0 for disabling.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use-itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### html-client
|
||||
|
||||
To experience it directly, open `html/static/index.html` in your browser. You will see the following page, which supports microphone input and file upload.
|
||||
<img src="images/html.png" width="900"/>
|
||||
|
||||
### java-client
|
||||
|
||||
```shell
|
||||
FunasrWsClient --host localhost --port 10095 --audio_in ./asr_example.wav --mode offline
|
||||
```
|
||||
For more details, please refer to the [docs](../java/readme.md)
|
||||
|
||||
## Server Usage Details
|
||||
|
||||
### Start the deployed FunASR service
|
||||
|
||||
If you have restarted the computer or shut down Docker after one-click deployment, you can start the FunASR service directly with the following command. The startup configuration is the same as the last one-click deployment.
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh start
|
||||
```
|
||||
|
||||
### Stop the FunASR service
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh stop
|
||||
```
|
||||
|
||||
### Release the FunASR service
|
||||
|
||||
Release the deployed FunASR service.
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh remove
|
||||
```
|
||||
|
||||
### Restart the FunASR service
|
||||
|
||||
Restart the FunASR service with the same configuration as the last one-click deployment.
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh restart
|
||||
```
|
||||
|
||||
### Replace the model and restart the FunASR service
|
||||
|
||||
Replace the currently used model, and restart the FunASR service. The model must be an ASR/VAD/PUNC model in ModelScope, or a finetuned model from ModelScope.
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update [--asr_model | --vad_model | --punc_model] <model_id or local model path>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update --asr_model damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch
|
||||
```
|
||||
|
||||
### Update parameters and restart the FunASR service
|
||||
|
||||
Update the configured parameters and restart the FunASR service to take effect. The parameters that can be updated include the host and Docker port numbers, as well as the number of inference and IO threads.
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update [--host_port | --docker_port] <port number>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update [--decode_thread_num | --io_thread_num] <the number of threads>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update [--workspace] <workspace in local>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update [--ssl] <0: close SSL; 1: open SSL, default:1>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update --decode_thread_num 32
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update --workspace /root/funasr-runtime-resources
|
||||
```
|
||||
|
||||
### Set SSL
|
||||
|
||||
SSL verification is enabled by default. If you need to disable it, you can set it when starting.
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update --ssl 0
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Contact Us
|
||||
|
||||
If you encounter any problems during use, please join our user group for feedback.
|
||||
|
||||
|
||||
| DingDing Group | Wechat |
|
||||
|:----------------------------------------------------------------------------:|:--------------------------------------------------------------:|
|
||||
| <div align="left"><img src="../../../docs/images/dingding.jpg" width="250"/> | <img src="../../../docs/images/wechat.png" width="232"/></div> |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
(简体中文|[English](./SDK_tutorial_en.md))
|
||||
|
||||
|
||||
# FunASR英文离线文件转写服务便捷部署教程
|
||||
FunASR提供可便捷本地或者云端服务器部署的离线文件转写服务，内核为FunASR已开源runtime-SDK。
|
||||
集成了达摩院语音实验室在Modelscope社区开源的语音端点检测(VAD)、Paraformer-large语音识别(ASR)、标点恢复(PUNC) 等相关能力，拥有完整的语音识别链路，可以将几十个小时的音频或视频识别成带标点的文字，而且支持上百路请求同时进行转写。
|
||||
|
||||
# 发布日志
|
||||
**FunASR英文离线文件转写服务1.0已发布，欢迎部署体验[快速上手](#快速上手)**
|
||||
|
||||
## 服务器配置
|
||||
|
||||
用户可以根据自己的业务需求，选择合适的服务器配置，推荐配置为：
|
||||
- 配置1: （X86，计算型），4核vCPU，内存8G，单机可以支持大约32路的请求
|
||||
- 配置2: （X86，计算型），16核vCPU，内存32G，单机可以支持大约64路的请求
|
||||
- 配置3: （X86，计算型），64核vCPU，内存128G，单机可以支持大约200路的请求
|
||||
|
||||
详细性能测试报告（[点击此处](./benchmark_onnx_cpp.md)）
|
||||
|
||||
云服务厂商，针对新用户，有3个月免费试用活动，申请教程（[点击此处](https://github.com/modelscope/FunASR/blob/main/runtime/docs/aliyun_server_tutorial.md)）
|
||||
|
||||
## 快速上手
|
||||
|
||||
### 服务端启动
|
||||
|
||||
`注意`：一键部署工具，过程分为：安装docker、下载docker镜像、启动服务。如果用户希望直接从FunASR docker镜像启动，可以参考开发指南（[点击此处](./SDK_advanced_guide_offline_en_zh.md)）
|
||||
|
||||
下载部署工具`funasr-runtime-deploy-offline-cpu-en.sh`
|
||||
|
||||
```shell
|
||||
curl -O https://raw.githubusercontent.com/alibaba-damo-academy/FunASR/main/runtime/deploy_tools/funasr-runtime-deploy-offline-cpu-en.sh;
|
||||
# 如遇到网络问题，中国大陆用户，可以使用下面的命令：
|
||||
# curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/funasr-runtime-deploy-offline-cpu-en.sh;
|
||||
```
|
||||
|
||||
执行部署工具，在提示处输入回车键即可完成服务端安装与部署。目前便捷部署工具暂时仅支持Linux环境，其他环境部署参考开发指南（[点击此处](./SDK_advanced_guide_offline_zh.md)）
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh install --workspace ./funasr-runtime-resources
|
||||
```
|
||||
|
||||
### 客户端测试与使用
|
||||
|
||||
运行上面安装指令后，会在/root/funasr-runtime-resources（默认安装目录）中下载客户端测试工具目录samples（手动下载，[点击此处](https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz)），
|
||||
我们以Python语言客户端为例，进行说明，支持多种音频格式输入（.wav, .pcm, .mp3等），也支持视频输入(.mp4等)，以及多文件列表wav.scp输入，其他版本客户端请参考文档（[点击此处](#客户端用法详解)）
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.wav"
|
||||
```
|
||||
|
||||
## 客户端用法详解
|
||||
|
||||
在服务器上完成FunASR服务部署以后，可以通过如下的步骤来测试和使用离线文件转写服务。
|
||||
目前分别支持以下几种编程语言客户端
|
||||
|
||||
- [Python](#python-client)
|
||||
- [CPP](#cpp-client)
|
||||
- [html](#html-client)
|
||||
- [java](#java-client)
|
||||
|
||||
更多版本客户端支持请参考[websocket/grpc协议](./websocket_protocol_zh.md)
|
||||
|
||||
### python-client
|
||||
若想直接运行client进行测试，可参考如下简易说明，以python版本为例：
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.wav"
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
```text
|
||||
--host 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10095 部署端口号
|
||||
--mode offline表示离线文件转写
|
||||
--audio_in 需要进行转写的音频文件，支持文件路径，文件列表wav.scp
|
||||
--thread_num 设置并发发送线程数，默认为1
|
||||
--ssl 设置是否开启ssl证书校验，默认1开启，设置为0关闭
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--use_itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### cpp-client
|
||||
进入samples/cpp目录后，可以用cpp进行测试，指令如下：
|
||||
```shell
|
||||
./funasr-wss-client --server-ip 127.0.0.1 --port 10095 --wav-path ../audio/asr_example.wav
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
|
||||
```text
|
||||
--server-ip 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10095 部署端口号
|
||||
--wav-path 需要进行转写的音频文件，支持文件路径
|
||||
--thread_num 设置并发发送线程数，默认为1
|
||||
--ssl 设置是否开启ssl证书校验，默认1开启，设置为0关闭
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--use-itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### html-client
|
||||
|
||||
在浏览器中打开 html/static/index.html，即可出现如下页面，支持麦克风输入与文件上传，直接进行体验
|
||||
|
||||
<img src="images/html.png" width="900"/>
|
||||
|
||||
### java-client
|
||||
|
||||
```shell
|
||||
FunasrWsClient --host localhost --port 10095 --audio_in ./asr_example.wav --mode offline
|
||||
```
|
||||
详细可以参考文档（[点击此处](../java/readme.md)）
|
||||
|
||||
## 服务端用法详解
|
||||
|
||||
### 启动已经部署过的FunASR服务
|
||||
一键部署后若出现重启电脑等关闭Docker的动作，可通过如下命令直接启动FunASR服务，启动配置为上次一键部署的设置。
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh start
|
||||
```
|
||||
|
||||
### 关闭FunASR服务
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh stop
|
||||
```
|
||||
|
||||
### 释放FunASR服务
|
||||
|
||||
释放已经部署的FunASR服务。
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh remove
|
||||
```
|
||||
|
||||
### 重启FunASR服务
|
||||
|
||||
根据上次一键部署的设置重启启动FunASR服务。
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh restart
|
||||
```
|
||||
|
||||
### 替换模型并重启FunASR服务
|
||||
|
||||
替换正在使用的模型，并重新启动FunASR服务。模型需为ModelScope中的ASR/VAD/PUNC模型，或者从ModelScope中模型finetune后的模型。
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update [--asr_model | --vad_model | --punc_model] <model_id or local model path>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update --asr_model damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch
|
||||
```
|
||||
|
||||
### 更新参数并重启FunASR服务
|
||||
|
||||
更新已配置参数，并重新启动FunASR服务生效。可更新参数包括宿主机和Docker的端口号，以及推理和IO的线程数量。
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update [--host_port | --docker_port] <port number>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update [--decode_thread_num | --io_thread_num] <the number of threads>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update [--workspace] <workspace in local>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update [--ssl] <0: close SSL; 1: open SSL, default:1>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update --decode_thread_num 32
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update --workspace /root/funasr-runtime-resources
|
||||
```
|
||||
|
||||
### 关闭SSL证书
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-en.sh update --ssl 0
|
||||
```
|
||||
|
||||
|
||||
|
||||
## 联系我们
|
||||
|
||||
在您使用过程中，如果遇到问题，欢迎加入用户群进行反馈
|
||||
|
||||
|
||||
| 钉钉用户群 | 微信 |
|
||||
|:----------------------------------------------------------------------------:|:-----------------------------------------------------:|
|
||||
| <div align="left"><img src="../../../docs/images/dingding.jpg" width="250"/> | <img src="../../../docs/images/wechat.png" width="232"/></div> |
|
||||
|
||||
|
||||
## 视频demo
|
||||
|
||||
[点击此处]()
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
([简体中文](./SDK_tutorial_online_zh.md)|English)
|
||||
|
||||
# FunASR Realtime Transcribe Service
|
||||
|
||||
FunASR offers a real-time speech-to-text service that can be easily deployed locally or on cloud servers. The service integrates various capabilities developed by the speech laboratory of DAMO Academy on the ModelScope, including voice activity detection (VAD), Paraformer-large non-streaming automatic speech recognition (ASR), Paraformer-large streaming ASR, and punctuation prediction (PUNC). The software package supports realtime speech-to-text service as well as high-precision transcription text correction at the end of each sentence and outputs text with punctuation.
|
||||
|
||||
## Server Configurations
|
||||
|
||||
Users can choose appropriate server configurations based on their business needs. The recommended configurations are:
|
||||
- Configuration 1: (X86, computing-type) 4-core vCPU, 8GB memory, and a single machine can support about 32 requests.
|
||||
- Configuration 2: (X86, computing-type) 16-core vCPU, 32GB memory, and a single machine can support about 64 requests.
|
||||
- Configuration 3: (X86, computing-type) 64-core vCPU, 128GB memory, and a single machine can support about 200 requests.
|
||||
|
||||
Detailed performance [report](./benchmark_onnx_cpp.md)
|
||||
|
||||
Cloud service providers offer a 3-month free trial for new users. Application tutorial ([docs](./aliyun_server_tutorial.md)).
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Server Startup
|
||||
|
||||
`Note`: The one-click deployment tool process includes installing Docker, downloading Docker images, and starting the service. If the user wants to start from the FunASR Docker image, please refer to the development guide ([docs](./SDK_advanced_guide_online.md).
|
||||
|
||||
Download the deployment tool `funasr-runtime-deploy-online-cpu-zh.sh`
|
||||
|
||||
```shell
|
||||
curl -O https://raw.githubusercontent.com/alibaba-damo-academy/FunASR/main/runtime/deploy_tools/funasr-runtime-deploy-online-cpu-zh.sh;
|
||||
# If there is a network problem, users in mainland China can use the following command:
|
||||
# curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/funasr-runtime-deploy-online-cpu-zh.sh;
|
||||
```
|
||||
|
||||
Execute the deployment tool and press the Enter key at the prompt to complete the installation and deployment of the server. Currently, the convenient deployment tool only supports Linux environments. For other environments, please refer to the development guide ([docs](./SDK_advanced_guide_online.md)).
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh install --workspace ./funasr-runtime-resources
|
||||
```
|
||||
Note: If you need to deploy the timestamp model or hotword model, select the corresponding model in step 2 of the installation and deployment process, where 1 is the paraformer-large model, 2 is the paraformer-large timestamp model, and 3 is the paraformer-large hotword model；The server loads the hotword file from: ./funasr-runtime-resources/hotowrds.txt (one hotword per line, formatted as word weight: 阿里巴巴 20).
|
||||
|
||||
### Client Testing and Usage
|
||||
|
||||
After running the above installation instructions, the client testing tool directory samples will be downloaded in the default installation directory ./funasr-runtime-resources ([download click](https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz)).
|
||||
We take the Python language client as an example to explain that it supports multiple audio format inputs (such as .wav, .pcm, .mp3, etc.), video inputs (.mp4, etc.), and multiple file list wav.scp inputs. For other client versions, please refer to the [documentation](#Detailed-Description-of-Client-Usage).
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode 2pass
|
||||
```
|
||||
|
||||
## Detailed Description of Client Usage
|
||||
|
||||
After completing the FunASR runtime-SDK service deployment on the server, you can test and use the offline file transcription service through the following steps. Currently, the following programming language client versions are supported:
|
||||
|
||||
- [Python](#python-client)
|
||||
- [CPP](#cpp-client)
|
||||
- [html](#html-client)
|
||||
- [java](#java-client)
|
||||
|
||||
For more client version support, please refer to the [websocket_protocol](./websocket_protocol_zh.md).
|
||||
|
||||
### python-client
|
||||
If you want to run the client directly for testing, you can refer to the following simple instructions, using the Python version as an example:
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.pcm"
|
||||
```
|
||||
|
||||
Command parameter instructions:
|
||||
```text
|
||||
--host is the IP address of the FunASR runtime-SDK service deployment machine, which defaults to the local IP address (127.0.0.1). If the client and the service are not on the same server, it needs to be changed to the deployment machine IP address.
|
||||
--port 10095 deployment port number
|
||||
--mode: `offline` indicates that the inference mode is one-sentence recognition; `online` indicates that the inference mode is real-time speech recognition; `2pass` indicates real-time speech recognition, and offline models are used for error correction at the end of each sentence.
|
||||
--chunk_size: indicates the latency configuration of the streaming model. [5,10,5] indicates that the current audio is 600ms, with a lookback of 300ms and a lookahead of 300ms.
|
||||
--audio_in is the audio file that needs to be transcribed, supporting file paths and file list wav.scp
|
||||
--thread_num sets the number of concurrent sending threads, default is 1
|
||||
--ssl sets whether to enable SSL certificate verification, default is 1 to enable, and 0 to disable
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use_itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### cpp-client
|
||||
|
||||
After entering the samples/cpp directory, you can test it with CPP. The command is as follows:
|
||||
```shell
|
||||
./funasr-wss-client-2pass --server-ip 127.0.0.1 --port 10095 --wav-path ../audio/asr_example.pcm
|
||||
```
|
||||
|
||||
Command parameter description:
|
||||
```text
|
||||
--server-ip specifies the IP address of the machine where the FunASR runtime-SDK service is deployed. The default value is the local IP address (127.0.0.1). If the client and the service are not on the same server, the IP address needs to be changed to the IP address of the deployment machine.
|
||||
--port specifies the deployment port number as 10095.
|
||||
--mode: `offline` indicates that the inference mode is one-sentence recognition; `online` indicates that the inference mode is real-time speech recognition; `2pass` indicates real-time speech recognition, and offline models are used for error correction at the end of each sentence.
|
||||
--chunk-size: indicates the latency configuration of the streaming model. [5,10,5] indicates that the current audio is 600ms, with a lookback of 300ms and a lookahead of 300ms.
|
||||
--record record is 1 means using record, fefault is 0
|
||||
--wav-path specifies the audio file to be transcribed, and supports file paths.
|
||||
--audio-fs the sample rate of the audio
|
||||
--threa-num sets the number of concurrent send threads, with a default value of 1.
|
||||
--is-ssl sets whether to enable SSL certificate verification, with a default value of 1 for enabling and 0 for disabling.
|
||||
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20)
|
||||
--use-itn: whether to use itn, the default value is 1 for enabling and 0 for disabling.
|
||||
```
|
||||
|
||||
### html-client
|
||||
|
||||
To experience it directly, open `html/static/index.html` in your browser. You will see the following page, which supports microphone input and file upload.
|
||||
<img src="images/html.png" width="900"/>
|
||||
|
||||
### java-client
|
||||
|
||||
```shell
|
||||
FunasrWsClient --host localhost --port 10095 --audio_in ./asr_example.wav --mode offline
|
||||
```
|
||||
For more details, please refer to the [docs](../java/readme.md)
|
||||
|
||||
## Server Usage Details
|
||||
|
||||
### Start the deployed FunASR service
|
||||
|
||||
If you have restarted the computer or shut down Docker after one-click deployment, you can start the FunASR service directly with the following command. The startup configuration is the same as the last one-click deployment.
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh start
|
||||
```
|
||||
|
||||
|
||||
|
||||
### Stop the FunASR service
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh stop
|
||||
```
|
||||
|
||||
### Release the FunASR service
|
||||
|
||||
Release the deployed FunASR service.
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh remove
|
||||
```
|
||||
|
||||
### Restart the FunASR service
|
||||
|
||||
Restart the FunASR service with the same configuration as the last one-click deployment.
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh restart
|
||||
```
|
||||
|
||||
### Replace the model and restart the FunASR service
|
||||
|
||||
Replace the currently used model, and restart the FunASR service. The model must be an ASR/VAD/PUNC model in ModelScope, or a finetuned model from ModelScope.
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update [--asr_model | --vad_model | --punc_model] <model_id or local model path>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update --asr_model damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch
|
||||
```
|
||||
|
||||
### Update parameters and restart the FunASR service
|
||||
|
||||
Update the configured parameters and restart the FunASR service to take effect. The parameters that can be updated include the host and Docker port numbers, as well as the number of inference and IO threads.
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update [--host_port | --docker_port] <port number>
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update [--decode_thread_num | --io_thread_num] <the number of threads>
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update [--workspace] <workspace in local>
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update [--ssl] <0: close SSL; 1: open SSL, default:1>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update --decode_thread_num 32
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update --workspace ./funasr-runtime-resources
|
||||
```
|
||||
|
||||
### Set SSL
|
||||
|
||||
SSL verification is enabled by default. If you need to disable it, you can set it when starting.
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update --ssl 0
|
||||
```
|
||||
|
||||
|
||||
## Contact Us
|
||||
|
||||
If you encounter any problems during use, please join our user group for feedback.
|
||||
|
||||
|
||||
| DingDing Group | Wechat |
|
||||
|:----------------------------------------------------------------------------:|:--------------------------------------------------------------:|
|
||||
| <div align="left"><img src="../../../docs/images/dingding.jpg" width="250"/> | <img src="../../../docs/images/wechat.png" width="232"/></div> |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,211 @@
|
||||
(简体中文|[English](./SDK_tutorial_online.md))
|
||||
|
||||
# FunASR实时语音听写便捷部署教程
|
||||
|
||||
FunASR提供可便捷本地或者云端服务器部署的实时语音听写服务，内核为FunASR已开源的runtime-SDK。
|
||||
FunASR集成了达摩院语音实验室在Modelscope社区开源的语音端点检测(VAD)、Paraformer-large非流式语音识别(ASR)、Paraformer-large流式语音识别(ASR)、标点预测(PUNC) 等相关能力。软件包既可以实时地进行语音转文字，而且能够在说话句尾用高精度的转写文字修正输出，输出文字带有标点，支持高并发多路请求
|
||||
|
||||
## 服务器配置
|
||||
|
||||
用户可以根据自己的业务需求，选择合适的服务器配置，推荐配置为：
|
||||
- 配置1: （X86，计算型），4核vCPU，内存8G，单机可以支持大约16路的请求
|
||||
- 配置2: （X86，计算型），16核vCPU，内存32G，单机可以支持大约32路的请求
|
||||
- 配置3: （X86，计算型），64核vCPU，内存128G，单机可以支持大约100路的请求
|
||||
|
||||
详细性能测试报告（[点击此处](./benchmark_onnx_cpp.md)）
|
||||
|
||||
云服务厂商，针对新用户，有3个月免费试用活动，申请教程（[点击此处](https://github.com/modelscope/FunASR/blob/main/runtime/docs/aliyun_server_tutorial.md)）
|
||||
|
||||
## 快速上手
|
||||
|
||||
### 服务端启动
|
||||
|
||||
`注意`：一键部署工具，过程分为：安装docker、下载docker镜像、启动服务。如果用户希望直接从FunASR docker镜像启动，可以参考开发指南（[点击此处](./SDK_advanced_guide_online_zh.md)）
|
||||
|
||||
下载部署工具`funasr-runtime-deploy-online-cpu-zh.sh`
|
||||
|
||||
```shell
|
||||
curl -O https://raw.githubusercontent.com/alibaba-damo-academy/FunASR/main/runtime/deploy_tools/funasr-runtime-deploy-online-cpu-zh.sh;
|
||||
# 如遇到网络问题，中国大陆用户，可以使用下面的命令：
|
||||
# curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/funasr-runtime-deploy-online-cpu-zh.sh;
|
||||
```
|
||||
|
||||
执行部署工具，在提示处输入回车键即可完成服务端安装与部署。目前便捷部署工具暂时仅支持Linux环境，其他环境部署参考开发指南（[点击此处](./SDK_advanced_guide_online_zh.md)）
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh install --workspace ./funasr-runtime-resources
|
||||
```
|
||||
**注：如果需要部署时间戳模型或者热词模型，在安装部署步骤2时选择对应模型，其中1为paraformer-large模型，2为paraformer-large 时间戳模型，3为paraformer-large nn热词模型；服务端加载热词文件地址为：./funasr-runtime-resources/hotowrds.txt(每行一个热词，格式为 热词 权重：阿里巴巴 20)**
|
||||
|
||||
### 客户端测试与使用
|
||||
|
||||
运行上面安装指令后，会在./funasr-runtime-resources（默认安装目录）中下载客户端测试工具目录samples（手动下载，[点击此处](https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz)），
|
||||
我们以Python语言客户端为例，支持麦克风输入，其他版本客户端请参考文档（[点击此处](#客户端用法详解)）
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10096 --mode 2pass
|
||||
```
|
||||
|
||||
## 客户端用法详解
|
||||
|
||||
在服务器上完成FunASR服务部署以后，可以通过如下的步骤来测试和使用实时语音听写服务。
|
||||
目前分别支持以下几种编程语言客户端
|
||||
|
||||
- [Python](#python-client)
|
||||
- [CPP](#cpp-client)
|
||||
- [html](#html-client)
|
||||
- [java](#java-client)
|
||||
- [c\#](#c\#)
|
||||
|
||||
更多版本客户端支持请参考[websocket/grpc协议](./websocket_protocol_zh.md)
|
||||
|
||||
### python-client
|
||||
若想直接运行client进行测试，可参考如下简易说明，以python版本为例：
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10096 --mode 2pass
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
```text
|
||||
--host 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10096 部署端口号
|
||||
--mode：`offline`表示推理模式为一句话识别；`online`表示推理模式为实时语音识别；`2pass`表示为实时语音识别，
|
||||
并且说话句尾采用离线模型进行纠错。
|
||||
--chunk_size：表示流式模型latency配置`[5,10,5]`，表示当前音频解码片段为600ms，并且回看300ms，右看300ms。
|
||||
--audio_in 需要进行转写的音频文件，支持文件路径，文件列表wav.scp
|
||||
--thread_num 设置并发发送线程数，默认为1
|
||||
--ssl 设置是否开启ssl证书校验，默认1开启，设置为0关闭+
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--use_itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### cpp-client
|
||||
进入samples/cpp目录后，可以用cpp进行测试，指令如下：
|
||||
```shell
|
||||
./funasr-wss-client-2pass --server-ip 127.0.0.1 --port 10096 --mode 2pass \
|
||||
--wav-path ../audio/asr_example.pcm
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
|
||||
```text
|
||||
--server-ip 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10096 部署端口号
|
||||
--mode：`offline`表示推理模式为一句话识别；`online`表示推理模式为实时语音识别；`2pass`表示为实时语音识别，
|
||||
并且说话句尾采用离线模型进行纠错。
|
||||
--chunk-size：表示流式模型latency配置`[5,10,5]`，表示当前音频解码片段为600ms，并且回看300ms，右看300ms。
|
||||
--record 1表示使用麦克风作为输入，默认为0
|
||||
--wav-path 需要进行转写的音频文件，支持文件路径
|
||||
--audio-fs pcm音频采样率
|
||||
--thread-num 设置并发发送线程数，默认为1
|
||||
--is-ssl 设置是否开启ssl证书校验，默认1开启，设置为0关闭
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--use-itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### html-client
|
||||
|
||||
在浏览器中打开 html/static/index.html，即可出现如下页面，支持麦克风输入与文件上传，直接进行体验
|
||||
|
||||
<img src="images/html.png" width="900"/>
|
||||
|
||||
### java-client
|
||||
|
||||
```shell
|
||||
FunasrWsClient --host localhost --port 10096 --mode 2pass
|
||||
```
|
||||
详细可以参考文档（[点击此处](../java/readme.md)）
|
||||
|
||||
## 服务端用法详解
|
||||
|
||||
### 启动已经部署过的FunASR服务
|
||||
一键部署后若出现重启电脑等关闭Docker的动作，可通过如下命令直接启动FunASR服务，启动配置为上次一键部署的设置。
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh start
|
||||
```
|
||||
|
||||
|
||||
|
||||
### 关闭FunASR服务
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh stop
|
||||
```
|
||||
|
||||
### 释放FunASR服务
|
||||
|
||||
释放已经部署的FunASR服务。
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh remove
|
||||
```
|
||||
|
||||
### 重启FunASR服务
|
||||
|
||||
根据上次一键部署的设置重启启动FunASR服务。
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh restart
|
||||
```
|
||||
|
||||
### 替换模型并重启FunASR服务
|
||||
|
||||
替换正在使用的模型，并重新启动FunASR服务。模型需为ModelScope中的ASR/VAD/PUNC模型，或者从ModelScope中模型finetune后的模型。
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update [--asr_model | --vad_model | --punc_model] <model_id or local model path>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update --asr_model damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch
|
||||
```
|
||||
|
||||
### 更新参数并重启FunASR服务
|
||||
|
||||
更新已配置参数，并重新启动FunASR服务生效。可更新参数包括宿主机和Docker的端口号，以及推理和IO的线程数量。
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update [--host_port | --docker_port] <port number>
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update [--decode_thread_num | --io_thread_num] <the number of threads>
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update [--workspace] <workspace in local>
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update [--ssl] <0: close SSL; 1: open SSL, default:1>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update --decode_thread_num 32
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update --workspace ./funasr-runtime-resources
|
||||
```
|
||||
|
||||
### 关闭SSL证书
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update --ssl 0
|
||||
```
|
||||
|
||||
## 联系我们
|
||||
|
||||
在您使用过程中，如果遇到问题，欢迎加入用户群进行反馈
|
||||
|
||||
|
||||
| 钉钉用户群 | 微信 |
|
||||
|:----------------------------------------------------------------------------:|:-----------------------------------------------------:|
|
||||
| <div align="left"><img src="../../../docs/images/dingding.jpg" width="250"/> | <img src="../../../docs/images/wechat.png" width="232"/></div> |
|
||||
|
||||
|
||||
## 视频demo
|
||||
|
||||
[点击此处]()
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,206 @@
|
||||
(简体中文|[English](./SDK_tutorial.md))
|
||||
|
||||
|
||||
# FunASR离线文件转写服务便捷部署教程
|
||||
FunASR提供可便捷本地或者云端服务器部署的离线文件转写服务，内核为FunASR已开源runtime-SDK。
|
||||
集成了达摩院语音实验室在Modelscope社区开源的语音端点检测(VAD)、Paraformer-large语音识别(ASR)、标点恢复(PUNC) 等相关能力，拥有完整的语音识别链路，可以将几十个小时的音频或视频识别成带标点的文字，而且支持上百路请求同时进行转写。
|
||||
|
||||
# 发布日志
|
||||
**FunASR离线文件转写服务已升级至2.0,集成ffmpeg支持多种音视频输入、支持热词模型、支持时间戳模型，欢迎部署体验[快速上手](#快速上手)**
|
||||
|
||||
## 服务器配置
|
||||
|
||||
用户可以根据自己的业务需求，选择合适的服务器配置，推荐配置为：
|
||||
- 配置1: （X86，计算型），4核vCPU，内存8G，单机可以支持大约32路的请求
|
||||
- 配置2: （X86，计算型），16核vCPU，内存32G，单机可以支持大约64路的请求
|
||||
- 配置3: （X86，计算型），64核vCPU，内存128G，单机可以支持大约200路的请求
|
||||
|
||||
详细性能测试报告（[点击此处](./benchmark_onnx_cpp.md)）
|
||||
|
||||
云服务厂商，针对新用户，有3个月免费试用活动，申请教程（[点击此处](https://github.com/modelscope/FunASR/blob/main/runtime/docs/aliyun_server_tutorial.md)）
|
||||
|
||||
## 快速上手
|
||||
|
||||
### 服务端启动
|
||||
|
||||
`注意`：一键部署工具，过程分为：安装docker、下载docker镜像、启动服务。如果用户希望直接从FunASR docker镜像启动，可以参考开发指南（[点击此处](./SDK_advanced_guide_offline_zh.md)）
|
||||
|
||||
下载部署工具`funasr-runtime-deploy-offline-cpu-zh.sh`
|
||||
|
||||
```shell
|
||||
curl -O https://raw.githubusercontent.com/alibaba-damo-academy/FunASR/main/runtime/deploy_tools/funasr-runtime-deploy-offline-cpu-zh.sh;
|
||||
# 如遇到网络问题，中国大陆用户，可以使用下面的命令：
|
||||
# curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/funasr-runtime-deploy-offline-cpu-zh.sh;
|
||||
```
|
||||
|
||||
执行部署工具，在提示处输入回车键即可完成服务端安装与部署。目前便捷部署工具暂时仅支持Linux环境，其他环境部署参考开发指南（[点击此处](./SDK_advanced_guide_offline_zh.md)）
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh install --workspace ./funasr-runtime-resources
|
||||
```
|
||||
**注：如果需要部署时间戳模型或者热词模型，在安装部署步骤2时选择对应模型，其中1为paraformer-large模型，2为paraformer-large 时间戳模型，3为paraformer-large nn热词模型；服务端加载热词文件地址为：./funasr-runtime-resources/hotowrds.txt(每行一个热词，格式为 热词 权重：阿里巴巴 20)**
|
||||
|
||||
|
||||
### 客户端测试与使用
|
||||
|
||||
运行上面安装指令后，会在/root/funasr-runtime-resources（默认安装目录）中下载客户端测试工具目录samples（手动下载，[点击此处](https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz)），
|
||||
我们以Python语言客户端为例，进行说明，支持多种音频格式输入（.wav, .pcm, .mp3等），也支持视频输入(.mp4等)，以及多文件列表wav.scp输入，其他版本客户端请参考文档（[点击此处](#客户端用法详解)）
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.wav"
|
||||
```
|
||||
|
||||
## 客户端用法详解
|
||||
|
||||
在服务器上完成FunASR服务部署以后，可以通过如下的步骤来测试和使用离线文件转写服务。
|
||||
目前分别支持以下几种编程语言客户端
|
||||
|
||||
- [Python](#python-client)
|
||||
- [CPP](#cpp-client)
|
||||
- [html](#html-client)
|
||||
- [java](#java-client)
|
||||
|
||||
更多版本客户端支持请参考[websocket/grpc协议](./websocket_protocol_zh.md)
|
||||
|
||||
### python-client
|
||||
若想直接运行client进行测试，可参考如下简易说明，以python版本为例：
|
||||
|
||||
```shell
|
||||
python3 funasr_wss_client.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.wav"
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
```text
|
||||
--host 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10095 部署端口号
|
||||
--mode offline表示离线文件转写
|
||||
--audio_in 需要进行转写的音频文件，支持文件路径，文件列表wav.scp
|
||||
--thread_num 设置并发发送线程数，默认为1
|
||||
--ssl 设置是否开启ssl证书校验，默认1开启，设置为0关闭
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--use_itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### cpp-client
|
||||
进入samples/cpp目录后，可以用cpp进行测试，指令如下：
|
||||
```shell
|
||||
./funasr-wss-client --server-ip 127.0.0.1 --port 10095 --wav-path ../audio/asr_example.wav
|
||||
```
|
||||
|
||||
命令参数说明：
|
||||
|
||||
```text
|
||||
--server-ip 为FunASR runtime-SDK服务部署机器ip，默认为本机ip（127.0.0.1），如果client与服务不在同一台服务器，
|
||||
需要改为部署机器ip
|
||||
--port 10095 部署端口号
|
||||
--wav-path 需要进行转写的音频文件，支持文件路径
|
||||
--thread_num 设置并发发送线程数，默认为1
|
||||
--ssl 设置是否开启ssl证书校验，默认1开启，设置为0关闭
|
||||
--hotword 热词文件，每行一个热词，格式(热词 权重)：阿里巴巴 20
|
||||
--use-itn 设置是否使用itn，默认1开启，设置为0关闭
|
||||
```
|
||||
|
||||
### html-client
|
||||
|
||||
在浏览器中打开 html/static/index.html，即可出现如下页面，支持麦克风输入与文件上传，直接进行体验
|
||||
|
||||
<img src="images/html.png" width="900"/>
|
||||
|
||||
### java-client
|
||||
|
||||
```shell
|
||||
FunasrWsClient --host localhost --port 10095 --audio_in ./asr_example.wav --mode offline
|
||||
```
|
||||
详细可以参考文档（[点击此处](../java/readme.md)）
|
||||
|
||||
## 服务端用法详解
|
||||
|
||||
### 启动已经部署过的FunASR服务
|
||||
一键部署后若出现重启电脑等关闭Docker的动作，可通过如下命令直接启动FunASR服务，启动配置为上次一键部署的设置。
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh start
|
||||
```
|
||||
|
||||
### 关闭FunASR服务
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh stop
|
||||
```
|
||||
|
||||
### 释放FunASR服务
|
||||
|
||||
释放已经部署的FunASR服务。
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh remove
|
||||
```
|
||||
|
||||
### 重启FunASR服务
|
||||
|
||||
根据上次一键部署的设置重启启动FunASR服务。
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh restart
|
||||
```
|
||||
|
||||
### 替换模型并重启FunASR服务
|
||||
|
||||
替换正在使用的模型，并重新启动FunASR服务。模型需为ModelScope中的ASR/VAD/PUNC模型，或者从ModelScope中模型finetune后的模型。
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update [--asr_model | --vad_model | --punc_model] <model_id or local model path>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update --asr_model damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch
|
||||
```
|
||||
|
||||
### 更新参数并重启FunASR服务
|
||||
|
||||
更新已配置参数，并重新启动FunASR服务生效。可更新参数包括宿主机和Docker的端口号，以及推理和IO的线程数量。
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update [--host_port | --docker_port] <port number>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update [--decode_thread_num | --io_thread_num] <the number of threads>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update [--workspace] <workspace in local>
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update [--ssl] <0: close SSL; 1: open SSL, default:1>
|
||||
|
||||
e.g
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update --decode_thread_num 32
|
||||
sudo bash funasr-runtime-deploy-offline-cpu-zh.sh update --workspace /root/funasr-runtime-resources
|
||||
```
|
||||
|
||||
### 关闭SSL证书
|
||||
|
||||
```shell
|
||||
sudo bash funasr-runtime-deploy-online-cpu-zh.sh update --ssl 0
|
||||
```
|
||||
|
||||
|
||||
|
||||
## 联系我们
|
||||
|
||||
在您使用过程中，如果遇到问题，欢迎加入用户群进行反馈
|
||||
|
||||
|
||||
| 钉钉用户群 | 微信 |
|
||||
|:----------------------------------------------------------------------------:|:-----------------------------------------------------:|
|
||||
| <div align="left"><img src="../../../docs/images/dingding.jpg" width="250"/> | <img src="../../../docs/images/wechat.png" width="232"/></div> |
|
||||
|
||||
|
||||
## 视频demo
|
||||
|
||||
[点击此处]()
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
# 云服务器申请教程
|
||||
|
||||
我们以阿里云（[点此链接](https://www.aliyun.com/)）为例，演示如何申请云服务器
|
||||
|
||||
### 登陆个人账号
|
||||
打开阿里云官网[点此链接](https://www.aliyun.com/)，注册并登陆个人账号，如下图标号1所示
|
||||
|
||||
<img src="images/aliyun1.png" width="900"/>
|
||||
|
||||
### 免费试用
|
||||
|
||||
点击如上图所以标号2，出现如下界面
|
||||
|
||||
<img src="images/aliyun2.png" width="900"/>
|
||||
|
||||
再点击标号3，出现如下界面
|
||||
|
||||
<img src="images/aliyun3.png" width="900"/>
|
||||
|
||||
### 申请ECS实例
|
||||
|
||||
个人账号可以免费试用1核2GB内存，每月750小时，企业认证后，可以免费试用2核8GB内存 3个月，根据账号情况，点击上图中标号4，出现如下界面：
|
||||
|
||||
<img src="images/aliyun4.png" width="900"/>
|
||||
|
||||
依次按照上图标号5、6、7选择后，点击立即试用，出现如下界面
|
||||
|
||||
<img src="images/aliyun5.png" width="900"/>
|
||||
|
||||
### 开放服务端口
|
||||
|
||||
点击安全组（标号9），出现如下界面
|
||||
|
||||
<img src="images/aliyun6.png" width="900"/>
|
||||
|
||||
再点击标号10，出现如下界面
|
||||
|
||||
<img src="images/aliyun7.png" width="900"/>
|
||||
|
||||
点击手动添加（标号11），分别按照标号12、13填入内容，后点击保存（标号14），再点击实例（标号15），出现如下界面
|
||||
|
||||
<img src="images/aliyun8.png" width="900"/>
|
||||
|
||||
### 启动ECS示例
|
||||
|
||||
点击示例名称（标号16），出现如下页面
|
||||
|
||||
<img src="images/aliyun9.png" width="900"/>
|
||||
|
||||
点击远程启动（标号17），出现页面后，点击立即登陆，出现如下界面
|
||||
|
||||
<img src="images/aliyun10.png" width="900"/>
|
||||
|
||||
首次登陆需要点击重置密码（上图中绿色箭头），设置好密码后，输入密码（标号18），点击确认（标号19）
|
||||
|
||||
<img src="images/aliyun11.png" width="900"/>
|
||||
|
||||
首次登陆会遇到上图所示问题，点击标号20，根据文档操作后，重新登陆，登陆成功后出现如下页面
|
||||
|
||||
<img src="images/aliyun12.png" width="900"/>
|
||||
|
||||
上图表示已经成功申请了云服务器，后续可以根据FunASR runtime-SDK部署文档进行一键部署（[点击此处](./SDK_tutorial_cn.md)）
|
||||
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# CPU Benchmark (Libtorch)
|
||||
|
||||
## Configuration
|
||||
### Data set:
|
||||
Aishell1 [test set](https://www.openslr.org/33/) , the total audio duration is 36108.919 seconds.
|
||||
|
||||
### Tools
|
||||
#### Install Requirements
|
||||
Install ModelScope and FunASR
|
||||
```shell
|
||||
pip install -U modelscope funasr
|
||||
# For the users in China, you could install with the command:
|
||||
#pip install -U funasr -i https://mirror.sjtu.edu.cn/pypi/web/simple
|
||||
```
|
||||
|
||||
Install requirements
|
||||
```shell
|
||||
git clone https://github.com/modelscope/FunASR.git && cd FunASR
|
||||
cd funasr/runtime/python/utils
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
#### Recipe
|
||||
|
||||
##### test_rtf
|
||||
set the model, data path and output_dir
|
||||
```shell
|
||||
nohup bash test_rtf.sh &> log.txt &
|
||||
```
|
||||
|
||||
##### test_cer
|
||||
set the model, data path and output_dir
|
||||
```shell
|
||||
nohup bash test_cer.sh &> log.txt &
|
||||
```
|
||||
|
||||
|
||||
## [Paraformer-large](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary)
|
||||
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz 16core-32processor with avx512_vnni
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|:----------------:|:------------------:|:------:|:------------:|
|
||||
| 1 (torch fp32) | 3522 | 0.0976 | 10.3 |
|
||||
| 1 (torch int8) | 1746 | 0.0484 | 20.7 |
|
||||
| 32 (torch fp32) | 236 | 0.0066 | 152.7 |
|
||||
| 32 (torch int8) | 114 | 0.0032 | 317.4 |
|
||||
| 64 (torch fp32) | 235 | 0.0065 | 153.7 |
|
||||
| 64 (torch int8) | 113 | 0.0031 | 319.2 |
|
||||
|
||||
|
||||
[//]: # (### Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz 32core-64processor without avx512_vnni)
|
||||
|
||||
|
||||
## [Paraformer](https://modelscope.cn/models/damo/speech_paraformer_asr_nat-zh-cn-16k-common-vocab8358-tensorflow1/summary)
|
||||
@@ -0,0 +1,33 @@
|
||||
# GPU Benchmark (libtorch-cpp)
|
||||
|
||||
## Configuration
|
||||
### Data set:
|
||||
A long audio test set(Non-open source) containing 103 audio files, with durations ranging from 2 to 30 minutes.
|
||||
|
||||
## [FSMN-VAD](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary) + [Paraformer-large](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-torchscript/summary) + [CT-Transformer](https://www.modelscope.cn/models/damo/punc_ct-transformer_zh-cn-common-vocab272727-onnx/summary)
|
||||
|
||||
```shell
|
||||
./funasr-onnx-offline-rtf \
|
||||
--model-dir ./damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-torchscript \
|
||||
--vad-dir ./damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir ./damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx \
|
||||
--gpu \
|
||||
--thread-num 20 \
|
||||
--bladedisc true \
|
||||
--batch-size 20 \
|
||||
--wav-path ./long_test.scp
|
||||
```
|
||||
Node: run in docker, ref to ([docs](./SDK_advanced_guide_offline_gpu_zh.md))
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni, GPU @ A10
|
||||
|
||||
| concurrent-tasks | batch | RTF | Speedup Rate |
|
||||
|------------------|:------:|:------:|:------------:|
|
||||
| 1 | 1 | 0.0076 | 130 |
|
||||
| 1 | 20 | 0.0048 | 208 |
|
||||
| 5 | 20 | 0.0011 | 850 |
|
||||
| 10 | 20 | 0.0008 | 1200+ |
|
||||
| 20 | 20 | 0.0008 | 1200+ |
|
||||
|
||||
Node: On CPUs, the single-thread RTF is 0.066, and 32-threads' speedup is 330+
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
# CPU Benchmark (ONNX-python)
|
||||
|
||||
## Configuration
|
||||
### Data set:
|
||||
Aishell1 [test set](https://www.openslr.org/33/) , the total audio duration is 36108.919 seconds.
|
||||
|
||||
### Tools
|
||||
#### Install Requirements
|
||||
Install ModelScope and FunASR
|
||||
```shell
|
||||
pip install -U modelscope funasr
|
||||
# For the users in China, you could install with the command:
|
||||
#pip install -U funasr -i https://mirror.sjtu.edu.cn/pypi/web/simple
|
||||
```
|
||||
|
||||
Install requirements
|
||||
```shell
|
||||
git clone https://github.com/modelscope/FunASR.git && cd FunASR
|
||||
cd funasr/runtime/python/utils
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
#### Recipe
|
||||
|
||||
|
||||
##### test_rtf
|
||||
set the model, data path and output_dir
|
||||
```shell
|
||||
nohup bash test_rtf.sh &> log.txt &
|
||||
```
|
||||
|
||||
##### test_cer
|
||||
set the model, data path and output_dir
|
||||
```shell
|
||||
nohup bash test_cer.sh &> log.txt &
|
||||
```
|
||||
|
||||
## [Paraformer-large](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary)
|
||||
|
||||
Number of Parameter: 220M
|
||||
|
||||
Storage size: 880MB
|
||||
|
||||
Storage size after int8-quant: 237MB
|
||||
|
||||
CER: 1.95%
|
||||
|
||||
CER after int8-quant: 1.95%
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|:----------------:|:------------------:|:-------:|:------------:|
|
||||
| 1 (onnx fp32) | 2806 | 0.0777 | 12.9 |
|
||||
| 1 (onnx int8) | 1611 | 0.0446 | 22.4 |
|
||||
| 8 (onnx fp32) | 538 | 0.0149 | 67.1 |
|
||||
| 8 (onnx int8) | 210 | 0.0058 | 172.4 |
|
||||
| 16 (onnx fp32) | 288 | 0.0080 | 125.2 |
|
||||
| 16 (onnx int8) | 117 | 0.0032 | 309.9 |
|
||||
| 32 (onnx fp32) | 167 | 0.0046 | 216.5 |
|
||||
| 32 (onnx int8) | 86 | 0.0024 | 420.0 |
|
||||
| 64 (onnx fp32) | 158 | 0.0044 | 228.1 |
|
||||
| 64 (onnx int8) | 82 | 0.0023 | 442.8 |
|
||||
| 96 (onnx fp32) | 151 | 0.0042 | 238.0 |
|
||||
| 96 (onnx int8) | 80 | 0.0022 | 452.0 |
|
||||
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz 16core-32processor with avx512_vnni
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|:----------------:|:------------------:|:------:|:------------:|
|
||||
| 1 (onnx fp32) | 2613 | 0.0724 | 13.8 |
|
||||
| 1 (onnx int8) | 1321 | 0.0366 | 22.4 |
|
||||
| 32 (onnx fp32) | 170 | 0.0047 | 212.7 |
|
||||
| 32 (onnx int8) | 89 | 0.0025 | 407.0 |
|
||||
| 64 (onnx fp32) | 166 | 0.0046 | 217.1 |
|
||||
| 64 (onnx int8) | 87 | 0.0024 | 414.7 |
|
||||
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz 32core-64processor without avx512_vnni
|
||||
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|:----------------:|:------------------:|:------:|:------------:|
|
||||
| 1 (onnx fp32) | 2959 | 0.0820 | 12.2 |
|
||||
| 1 (onnx int8) | 2814 | 0.0778 | 12.8 |
|
||||
| 16 (onnx fp32) | 373 | 0.0103 | 96.9 |
|
||||
| 16 (onnx int8) | 331 | 0.0091 | 109.0 |
|
||||
| 32 (onnx fp32) | 211 | 0.0058 | 171.4 |
|
||||
| 32 (onnx int8) | 181 | 0.0050 | 200.0 |
|
||||
| 64 (onnx fp32) | 153 | 0.0042 | 235.9 |
|
||||
| 64 (onnx int8) | 103 | 0.0029 | 349.9 |
|
||||
| 96 (onnx fp32) | 146 | 0.0041 | 247.0 |
|
||||
| 96 (onnx int8) | 108 | 0.0030 | 334.1 |
|
||||
|
||||
## [Paraformer](https://modelscope.cn/models/damo/speech_paraformer_asr_nat-zh-cn-16k-common-vocab8358-tensorflow1/summary)
|
||||
|
||||
Number of Parameter: 68M
|
||||
|
||||
Storage size: 275MB
|
||||
|
||||
Storage size after int8-quant: 81MB
|
||||
|
||||
CER: 3.73%
|
||||
|
||||
CER after int8-quant: 3.78%
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|:----------------:|:------------------:|:------:|:------------:|
|
||||
| 1 (onnx fp32) | 1173 | 0.0325 | 30.8 |
|
||||
| 1 (onnx int8) | 976 | 0.0270 | 37.0 |
|
||||
| 16 (onnx fp32) | 91 | 0.0025 | 395.2 |
|
||||
| 16 (onnx int8) | 78 | 0.0022 | 463.0 |
|
||||
| 32 (onnx fp32) | 60 | 0.0017 | 598.8 |
|
||||
| 32 (onnx int8) | 40 | 0.0011 | 892.9 |
|
||||
| 64 (onnx fp32) | 55 | 0.0015 | 653.6 |
|
||||
| 64 (onnx int8) | 31 | 0.0009 | 1162.8 |
|
||||
| 96 (onnx fp32) | 57 | 0.0016 | 632.9 |
|
||||
| 96 (onnx int8) | 33 | 0.0009 | 1098.9 |
|
||||
@@ -0,0 +1,250 @@
|
||||
# CPU Benchmark (ONNX-cpp)
|
||||
|
||||
## Configuration
|
||||
### Data set:
|
||||
Aishell1 [test set](https://www.openslr.org/33/) , the total audio duration is 36108.919 seconds.
|
||||
|
||||
### Tools
|
||||
#### Install [modelscope and funasr](https://github.com/modelscope/FunASR#installation)
|
||||
|
||||
```shell
|
||||
pip3 install torch torchaudio
|
||||
pip install -U modelscope
|
||||
pip install -U funasr
|
||||
```
|
||||
|
||||
#### Export [onnx model](https://github.com/modelscope/FunASR/tree/main/funasr/export)
|
||||
|
||||
```shell
|
||||
python -m funasr.export.export_model --model-name damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch --export-dir ./export --type onnx --quantize True
|
||||
```
|
||||
|
||||
#### Building for Linux/Unix
|
||||
|
||||
Download onnxruntime
|
||||
```shell
|
||||
# download an appropriate onnxruntime from https://github.com/microsoft/onnxruntime/releases/tag/v1.14.0
|
||||
# here we get a copy of onnxruntime for linux 64
|
||||
wget https://github.com/microsoft/onnxruntime/releases/download/v1.14.0/onnxruntime-linux-x64-1.14.0.tgz
|
||||
tar -zxvf onnxruntime-linux-x64-1.14.0.tgz
|
||||
```
|
||||
|
||||
Install openblas
|
||||
```shell
|
||||
sudo apt-get install libopenblas-dev #ubuntu
|
||||
# sudo yum -y install openblas-devel #centos
|
||||
```
|
||||
|
||||
Build runtime
|
||||
```shell
|
||||
git clone https://github.com/modelscope/FunASR.git && cd funasr/runtime/onnxruntime
|
||||
mkdir build && cd build
|
||||
cmake -DCMAKE_BUILD_TYPE=release .. -DONNXRUNTIME_DIR=/path/to/onnxruntime-linux-x64-1.14.0
|
||||
make
|
||||
```
|
||||
|
||||
## [Paraformer-large](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary)
|
||||
```shell
|
||||
./funasr-onnx-offline-rtf \
|
||||
--model-dir ./damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch \
|
||||
--quantize true \
|
||||
--wav-path ./aishell1_test.scp \
|
||||
--thread-num 32
|
||||
|
||||
Node: '--quantize false' means fp32, otherwise it will be int8
|
||||
```
|
||||
|
||||
Number of Parameter: 220M
|
||||
|
||||
Storage size: 880MB
|
||||
|
||||
Storage size after int8-quant: 237MB
|
||||
|
||||
CER: 1.95%
|
||||
|
||||
CER after int8-quant: 1.95%
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|---------------------|:------------------:|:--------:|:------------:|
|
||||
| 1 (onnx fp32) | 2129s | 0.058974 | 17 |
|
||||
| 1 (onnx int8) | 1020s | 0.02826 | 35 |
|
||||
| 8 (onnx fp32) | 273s | 0.007553 | 132 |
|
||||
| 8 (onnx int8) | 128s | 0.003558 | 281 |
|
||||
| 16 (onnx fp32) | 146s | 0.00403 | 248 |
|
||||
| 16 (onnx int8) | 67s | 0.001868 | 535 |
|
||||
| 32 (onnx fp32) | 133s | 0.003672 | 272 |
|
||||
| 32 (onnx int8) | 64s | 0.001778 | 562 |
|
||||
| 64 (onnx fp32) | 136s | 0.003771 | 265 |
|
||||
| 64 (onnx int8) | 67s | 0.001846 | 541 |
|
||||
| 96 (onnx fp32) | 137s | 0.003788 | 264 |
|
||||
| 96 (onnx int8) | 68s | 0.001875 | 533 |
|
||||
|
||||
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz 32core-64processor without avx512_vnni
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|---------------------|:------------------:|----------|:------------:|
|
||||
| 1 (onnx fp32) | 2903s | 0.080404 | 12 |
|
||||
| 1 (onnx int8) | 2714s | 0.075168 | 13 |
|
||||
| 8 (onnx fp32) | 373s | 0.010329 | 97 |
|
||||
| 8 (onnx int8) | 340s | 0.009428 | 106 |
|
||||
| 16 (onnx fp32) | 189s | 0.005252 | 190 |
|
||||
| 16 (onnx int8) | 174s | 0.004817 | 207 |
|
||||
| 32 (onnx fp32) | 109s | 0.00301 | 332 |
|
||||
| 32 (onnx int8) | 88s | 0.00245 | 408 |
|
||||
| 64 (onnx fp32) | 113s | 0.003129 | 320 |
|
||||
| 64 (onnx int8) | 79s | 0.002201 | 454 |
|
||||
| 96 (onnx fp32) | 115s | 0.003183 | 314 |
|
||||
| 96 (onnx int8) | 80s | 0.002222 | 450 |
|
||||
|
||||
## [FSMN-VAD](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary) + [Paraformer-large](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary) + [CT-Transformer](https://www.modelscope.cn/models/damo/punc_ct-transformer_zh-cn-common-vocab272727-onnx/summary)
|
||||
|
||||
```shell
|
||||
./funasr-onnx-offline-rtf \
|
||||
--model-dir ./damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch \
|
||||
--quantize true \
|
||||
--vad-dir ./damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir ./damo/punc_ct-transformer_zh-cn-common-vocab272727-onnx \
|
||||
--wav-path ./aishell1_test.scp \
|
||||
--thread-num 32
|
||||
|
||||
Node: '--quantize false' means fp32, otherwise it will be int8
|
||||
```
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|---------------------|:------------------:|:--------:|:------------:|
|
||||
| 1 (onnx fp32) | 2134s | 0.0591 | 17 |
|
||||
| 1 (onnx int8) | 1047s | 0.029 | 34 |
|
||||
| 8 (onnx fp32) | 273s | 0.007557 | 132 |
|
||||
| 8 (onnx int8) | 132s | 0.003647 | 274 |
|
||||
| 16 (onnx fp32) | 147s | 0.004061 | 246 |
|
||||
| 16 (onnx int8) | 69s | 0.001916 | 521 |
|
||||
| 32 (onnx fp32) | 133s | 0.003675 | 272 |
|
||||
| 32 (onnx int8) | 65s | 0.001786 | 559 |
|
||||
| 64 (onnx fp32) | 136s | 0.003767 | 265 |
|
||||
| 64 (onnx int8) | 67s | 0.001867 | 535 |
|
||||
| 96 (onnx fp32) | 137s | 0.003802 | 262 |
|
||||
| 96 (onnx int8) | 69s | 0.001904 | 524 |
|
||||
|
||||
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz 32core-64processor without avx512_vnni
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|---------------------|:------------------:|----------|:------------:|
|
||||
| 1 (onnx fp32) | 3073s | 0.0851 | 12 |
|
||||
| 1 (onnx int8) | 2840s | 0.0787 | 13 |
|
||||
| 8 (onnx fp32) | 389s | 0.01079 | 93 |
|
||||
| 8 (onnx int8) | 355s | 0.0098 | 101 |
|
||||
| 16 (onnx fp32) | 199s | 0.005513 | 181 |
|
||||
| 16 (onnx int8) | 171s | 0.004784 | 210 |
|
||||
| 32 (onnx fp32) | 113s | 0.00314 | 318 |
|
||||
| 32 (onnx int8) | 92s | 0.00255 | 391 |
|
||||
| 64 (onnx fp32) | 115s | 0.0032 | 312 |
|
||||
| 64 (onnx int8) | 81s | 0.002232 | 448 |
|
||||
| 96 (onnx fp32) | 117s | 0.003257 | 307 |
|
||||
| 96 (onnx int8) | 81s | 0.002258 | 442 |
|
||||
|
||||
|
||||
|
||||
## [FSMN-VAD](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary) + [Paraformer-large](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary) +[Ngram](https://www.modelscope.cn/models/damo/speech_ngram_lm_zh-cn-ai-wesp-fst/summary) + [CT-Transformer](https://www.modelscope.cn/models/damo/punc_ct-transformer_zh-cn-common-vocab272727-onnx/summary)
|
||||
|
||||
```shell
|
||||
./funasr-onnx-offline-rtf \
|
||||
--model-dir ./damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch \
|
||||
--quantize true \
|
||||
--vad-dir ./damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir ./damo/punc_ct-transformer_zh-cn-common-vocab272727-onnx \
|
||||
--lm-dir ./damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--wav-path ./aishell1_test.scp \
|
||||
--thread-num 32
|
||||
|
||||
Node: '--quantize false' means fp32, otherwise it will be int8
|
||||
```
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|------------------|:------------------:|:------:|:------------:|
|
||||
| 1 (onnx fp32) | 2506s | 0.0694 | 14 |
|
||||
| 1 (onnx int8) | 1448s | 0.0401 | 25 |
|
||||
| 8 (onnx fp32) | 326s | 0.0090 | 110 |
|
||||
| 8 (onnx int8) | 184s | 0.0051 | 195 |
|
||||
| 16 (onnx fp32) | 178s | 0.0049 | 202 |
|
||||
| 16 (onnx int8) | 99s | 0.0027 | 361 |
|
||||
| 32 (onnx fp32) | 152s | 0.0042 | 236 |
|
||||
| 32 (onnx int8) | 85s | 0.0023 | 422 |
|
||||
| 64 (onnx fp32) | 157s | 0.0043 | 228 |
|
||||
| 64 (onnx int8) | 89s | 0.0024 | 403 |
|
||||
| 96 (onnx fp32) | 158s | 0.0044 | 227 |
|
||||
| 96 (onnx int8) | 91s | 0.0025 | 396 |
|
||||
|
||||
|
||||
```shell
|
||||
# using hotwords
|
||||
./funasr-onnx-offline-rtf \
|
||||
--model-dir ./damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch \
|
||||
--quantize true \
|
||||
--vad-dir ./damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir ./damo/punc_ct-transformer_zh-cn-common-vocab272727-onnx \
|
||||
--lm-dir ./damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
|
||||
--hotword hotwords_dev_600.txt \
|
||||
--wav-path ./aishell1_test.scp \
|
||||
--thread-num 32
|
||||
|
||||
Node: '--quantize false' means fp32, otherwise it will be int8
|
||||
```
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|------------------|:------------------:|:------:|:------------:|
|
||||
| 1 (onnx fp32) | 3172s | 0.0878 | 11 |
|
||||
| 1 (onnx int8) | 2140s | 0.0592 | 16 |
|
||||
| 8 (onnx fp32) | 412s | 0.0114 | 87 |
|
||||
| 8 (onnx int8) | 268s | 0.0074 | 134 |
|
||||
| 16 (onnx fp32) | 218s | 0.0060 | 165 |
|
||||
| 16 (onnx int8) | 140s | 0.0038 | 257 |
|
||||
| 32 (onnx fp32) | 183s | 0.0050 | 196 |
|
||||
| 32 (onnx int8) | 116s | 0.0032 | 310 |
|
||||
| 64 (onnx fp32) | 188s | 0.0052 | 192 |
|
||||
| 64 (onnx int8) | 120s | 0.0033 | 299 |
|
||||
| 96 (onnx fp32) | 191s | 0.0052 | 188 |
|
||||
| 96 (onnx int8) | 122s | 0.0033 | 294 |
|
||||
|
||||
|
||||
## [FSMN-VAD](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary) + [Paraformer-en](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-en-16k-common-vocab10020-onnx/summary) + [CT-Transformer](https://www.modelscope.cn/models/damo/punc_ct-transformer_zh-cn-common-vocab272727-onnx/summary)
|
||||
|
||||
```shell
|
||||
./funasr-onnx-offline-rtf \
|
||||
--model-dir ./damo/speech_paraformer-large_asr_nat-en-16k-common-vocab10020-onnx \
|
||||
--quantize true \
|
||||
--vad-dir ./damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
||||
--punc-dir ./damo/punc_ct-transformer_zh-cn-common-vocab272727-onnx \
|
||||
--wav-path ./librispeech_test_clean.scp \
|
||||
--thread-num 32
|
||||
|
||||
Node: '--quantize false' means fp32, otherwise it will be int8
|
||||
```
|
||||
|
||||
### Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni
|
||||
|
||||
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
||||
|---------------------|:------------------:|----------|:------------:|
|
||||
| 1 (onnx fp32) | 1327s | 0.0682 | 15 |
|
||||
| 1 (onnx int8) | 734s | 0.0377 | 26 |
|
||||
| 8 (onnx fp32) | 169s | 0.0087 | 114 |
|
||||
| 8 (onnx int8) | 94s | 0.0048 | 205 |
|
||||
| 16 (onnx fp32) | 89s | 0.0046 | 217 |
|
||||
| 16 (onnx int8) | 50s | 0.0025 | 388 |
|
||||
| 32 (onnx fp32) | 78s | 0.0040 | 248 |
|
||||
| 32 (onnx int8) | 43s | 0.0022 | 448 |
|
||||
| 64 (onnx fp32) | 79s | 0.0041 | 243 |
|
||||
| 64 (onnx int8) | 44s | 0.0022 | 438 |
|
||||
| 96 (onnx fp32) | 80s | 0.0041 | 240 |
|
||||
| 96 (onnx int8) | 45s | 0.0023 | 428 |
|
||||
@@ -0,0 +1,10 @@
|
||||
DOCKER:
|
||||
funasr-runtime-sdk-en-cpu-0.1.6
|
||||
funasr-runtime-sdk-en-cpu-0.1.5
|
||||
funasr-runtime-sdk-en-cpu-0.1.4
|
||||
DEFAULT_ASR_MODEL:
|
||||
damo/speech_paraformer-large_asr_nat-en-16k-common-vocab10020-onnx
|
||||
DEFAULT_VAD_MODEL:
|
||||
damo/speech_fsmn_vad_zh-cn-16k-common-onnx
|
||||
DEFAULT_PUNC_MODEL:
|
||||
damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx
|
||||
@@ -0,0 +1,15 @@
|
||||
DOCKER:
|
||||
funasr-runtime-sdk-cpu-0.4.5
|
||||
funasr-runtime-sdk-cpu-0.3.0
|
||||
funasr-runtime-sdk-cpu-0.2.2
|
||||
DEFAULT_ASR_MODEL:
|
||||
damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
damo/speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
DEFAULT_VAD_MODEL:
|
||||
damo/speech_fsmn_vad_zh-cn-16k-common-onnx
|
||||
DEFAULT_PUNC_MODEL:
|
||||
damo/punc_ct-transformer_cn-en-common-vocab471067-large-onnx
|
||||
DEFAULT_LM_MODEL:
|
||||
damo/speech_ngram_lm_zh-cn-ai-wesp-fst
|
||||
NONE
|
||||
@@ -0,0 +1,14 @@
|
||||
DOCKER:
|
||||
funasr-runtime-sdk-online-cpu-0.1.10
|
||||
funasr-runtime-sdk-online-cpu-0.1.9
|
||||
funasr-runtime-sdk-online-cpu-0.1.8
|
||||
DEFAULT_ASR_MODEL:
|
||||
damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
damo/speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404-onnx
|
||||
DEFAULT_ASR_ONLINE_MODEL:
|
||||
damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx
|
||||
DEFAULT_VAD_MODEL:
|
||||
damo/speech_fsmn_vad_zh-cn-16k-common-onnx
|
||||
DEFAULT_PUNC_MODEL:
|
||||
damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 144 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.4 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.9 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.8 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.8 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.0 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.6 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 3.2 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.1 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.0 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.6 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.3 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.5 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.7 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 270 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 86 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 869 KiB |
@@ -0,0 +1,43 @@
|
||||
# 如何训练LM
|
||||
|
||||
训练脚本详见（[点击此处](../tools/train_compile_ngram.sh)）
|
||||
|
||||
## 数据准备
|
||||
```shell
|
||||
# 下载: 示例训练语料text、lexicon 和 am建模单元units.txt
|
||||
wget https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/requirements/lm.tar.gz
|
||||
# 如果是匹配8k的am模型，使用 https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/requirements/lm_8358.tar.gz
|
||||
tar -zxvf lm.tar.gz
|
||||
```
|
||||
|
||||
## 训练arpa
|
||||
```shell
|
||||
# make sure that srilm is installed
|
||||
# the format of the text should be:
|
||||
# BAC009S0002W0122 而 对 楼市 成交 抑制 作用 最 大 的 限 购
|
||||
# BAC009S0002W0123 也 成为 地方 政府 的 眼中 钉
|
||||
|
||||
bash fst/train_lms.sh
|
||||
```
|
||||
|
||||
## 生成lexicon
|
||||
```shell
|
||||
python3 fst/generate_lexicon.py lm/corpus.dict lm/lexicon.txt lm/lexicon.out
|
||||
```
|
||||
|
||||
## 编译TLG.fst
|
||||
编译TLG需要依赖fst的环境，请参考文档安装fts相关环境（[点击此处](../onnxruntime/readme.md)）
|
||||
```shell
|
||||
|
||||
# Compile the lexicon and token FSTs
|
||||
fst/compile_dict_token.sh lm lm/tmp lm/lang
|
||||
|
||||
# Compile the language-model FST and the final decoding graph TLG.fst
|
||||
fst/make_decode_graph.sh lm lm/lang || exit 1;
|
||||
|
||||
# Collect resource files required for decoding
|
||||
fst/collect_resource_file.sh lm lm/resource
|
||||
|
||||
#编译后的模型资源位于 lm/resource
|
||||
```
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
([简体中文](./websocket_protocol_zh.md)|English)
|
||||
|
||||
# WebSocket/gRPC Communication Protocol
|
||||
This protocol is the communication protocol for the FunASR software package, which includes offline file transcription ([deployment document](./SDK_tutorial.md)) and real-time speech recognition ([deployment document](./SDK_tutorial_online.md)).
|
||||
|
||||
## Offline File Transcription
|
||||
### Sending Data from Client to Server
|
||||
#### Message Format
|
||||
Configuration parameters and meta information are in JSON format, while audio data is in bytes.
|
||||
#### Initial Communication
|
||||
The message (which needs to be serialized in JSON) is:
|
||||
```text
|
||||
{"mode": "offline", "wav_name": "wav_name", "wav_format":"pcm", "is_speaking": True, "wav_format":"pcm", "hotwords":"{"阿里巴巴":20,"通义实验室":30}", "itn":True}
|
||||
```
|
||||
Parameter explanation:
|
||||
```text
|
||||
`mode`: `offline`, indicating the inference mode for offline file transcription
|
||||
`wav_name`: the name of the audio file to be transcribed
|
||||
`wav_format`: the audio and video file extension, such as pcm, mp3, mp4, etc.
|
||||
`is_speaking`: False indicates the end of a sentence, such as a VAD segmentation point or the end of a WAV file
|
||||
`audio_fs`: when the input audio is in PCM format, the audio sampling rate parameter needs to be added
|
||||
`hotwords`：If using the hotword, you need to send the hotword data (string) to the server. For example："{"阿里巴巴":20,"通义实验室":30}"
|
||||
`itn`: whether to use itn, the default value is true for enabling and false for disabling.
|
||||
```
|
||||
|
||||
#### Sending Audio Data
|
||||
For PCM format, directly send the audio data. For other audio formats, send the header information and audio and video bytes data together. Multiple sampling rates and audio and video formats are supported.
|
||||
|
||||
#### Sending End of Audio Flag
|
||||
After sending the audio data, an end-of-audio flag needs to be sent (which needs to be serialized in JSON):
|
||||
```text
|
||||
{"is_speaking": False}
|
||||
```
|
||||
|
||||
### Sending Data from Server to Client
|
||||
#### Sending Recognition Results
|
||||
The message (serialized in JSON) is:
|
||||
```text
|
||||
{"mode": "offline", "wav_name": "wav_name", "text": "asr ouputs", "is_final": True, "timestamp":"[[100,200], [200,500]]", "stamp_sents":[]}
|
||||
```
|
||||
Parameter explanation:
|
||||
```text
|
||||
`mode`: `offline`, indicating the inference mode for offline file transcription
|
||||
`wav_name`: the name of the audio file to be transcribed
|
||||
`text`: the text output of speech recognition
|
||||
`is_final`: indicating the end of recognition
|
||||
`timestamp`：If AM is a timestamp model, it will return this field, indicating the timestamp, in the format of "[[100,200], [200,500]]"
|
||||
`stamp_sents`：If AM is a timestamp model, it will return this field, indicating the stamp_sents, in the format of [{"text_seg":"正 是 因 为","punc":",","start":430,"end":1130,"ts_list":[[430,670],[670,810],[810,1030],[1030,1130]]}]
|
||||
```
|
||||
|
||||
## Real-time Speech Recognition
|
||||
### System Architecture Diagram
|
||||
|
||||
<div align="left"><img src="images/2pass.jpg" width="600"/></div>
|
||||
|
||||
### Sending Data from Client to Server
|
||||
#### Message Format
|
||||
Configuration parameters and meta information are in JSON format, while audio data is in bytes.
|
||||
|
||||
#### Initial Communication
|
||||
The message (which needs to be serialized in JSON) is:
|
||||
```text
|
||||
{"mode": "2pass", "wav_name": "wav_name", "is_speaking": True, "wav_format":"pcm", "chunk_size":[5,10,5],"hotwords":"{"阿里巴巴":20,"通义实验室":30}","itn":true}
|
||||
```
|
||||
Parameter explanation:
|
||||
```text
|
||||
`mode`: `offline` indicates the inference mode for single-sentence recognition; `online` indicates the inference mode for real-time speech recognition; `2pass` indicates real-time speech recognition and offline model correction for sentence endings.
|
||||
`wav_name`: the name of the audio file to be transcribed
|
||||
`wav_format`: the audio and video file extension, such as pcm, mp3, mp4, etc. (Note: only PCM audio streams are supported in version 1.0)
|
||||
`is_speaking`: False indicates the end of a sentence, such as a VAD segmentation point or the end of a WAV file
|
||||
`chunk_size`: indicates the latency configuration of the streaming model, `[5,10,5]` indicates that the current audio is 600ms long, with a 300ms look-ahead and look-back time.
|
||||
`audio_fs`: when the input audio is in PCM format, the audio sampling rate parameter needs to be added
|
||||
`hotwords`：If using the hotword, you need to send the hotword data (string) to the server. For example："{"阿里巴巴":20,"通义实验室":30}"
|
||||
`itn`: whether to use itn, the default value is true for enabling and false for disabling.
|
||||
```
|
||||
#### Sending Audio Data
|
||||
Directly send the audio data, removing the header information and sending only the bytes data. Supported audio sampling rates are 8000 (which needs to be specified as audio_fs in message), and 16000.
|
||||
#### Sending End of Audio Flag
|
||||
After sending the audio data, an end-of-audio flag needs to be sent (which needs to be serialized in JSON):
|
||||
```text
|
||||
{"is_speaking": False}
|
||||
```
|
||||
### Sending Data from Server to Client
|
||||
#### Sending Recognition Results
|
||||
The message (serialized in JSON) is:
|
||||
|
||||
```text
|
||||
{"mode": "2pass-online", "wav_name": "wav_name", "text": "asr ouputs", "is_final": True, "timestamp":"[[100,200], [200,500]]", "stamp_sents":[]}
|
||||
```
|
||||
Parameter explanation:
|
||||
```text
|
||||
`mode`: indicates the inference mode, divided into `2pass-online` for real-time recognition results and `2pass-offline` for 2-pass corrected recognition results.
|
||||
`wav_name`: the name of the audio file to be transcribed
|
||||
`text`: the text output of speech recognition
|
||||
`is_final`: indicating the end of recognition
|
||||
`timestamp`：If AM is a timestamp model, it will return this field, indicating the timestamp, in the format of "[[100,200], [200,500]]"
|
||||
`stamp_sents`：If AM is a timestamp model, it will return this field, indicating the stamp_sents, in the format of [{"text_seg":"正 是 因 为","punc":",","start":430,"end":1130,"ts_list":[[430,670],[670,810],[810,1030],[1030,1130]]}]
|
||||
```
|
||||
@@ -0,0 +1,104 @@
|
||||
(简体中文|[English](./websocket_protocol.md))
|
||||
# websocket/grpc通信协议
|
||||
|
||||
本协议为FunASR软件包通信协议，分为离线文件转写（[部署文档](./SDK_tutorial_zh.md)），实时语音识别（[部署文档](./SDK_tutorial_online_zh.md)）
|
||||
|
||||
## 离线文件转写
|
||||
### 从客户端往服务端发送数据
|
||||
#### 消息格式
|
||||
配置参数与meta信息用json，音频数据采用bytes
|
||||
#### 首次通信
|
||||
message为（需要用json序列化）：
|
||||
```text
|
||||
{"mode": "offline", "wav_name": "wav_name", "wav_format":"pcm", "is_speaking": True, "hotwords":"{"阿里巴巴":20,"通义实验室":30}", "itn":True}
|
||||
```
|
||||
参数介绍：
|
||||
```text
|
||||
`mode`：`offline`，表示推理模式为离线文件转写
|
||||
`wav_name`：表示需要推理音频文件名
|
||||
`wav_format`：表示音视频文件后缀名，可选pcm、mp3、mp4等
|
||||
`is_speaking`：False 表示断句尾点，例如，vad切割点，或者一条wav结束
|
||||
`audio_fs`：当输入音频为pcm数据时，需要加上音频采样率参数
|
||||
`hotwords`：如果使用热词，需要向服务端发送热词数据（字符串），格式为 "{"阿里巴巴":20,"通义实验室":30}"
|
||||
`itn`: 设置是否使用itn，默认True
|
||||
`svs_lang`: 设置SenseVoiceSmall模型语种，默认为“auto”
|
||||
`svs_itn`: 设置SenseVoiceSmall模型是否开启标点、ITN，默认为True
|
||||
```
|
||||
注：热词权重仅在fst热词服务下生效。
|
||||
|
||||
#### 发送音频数据
|
||||
pcm直接将音频数据，其他格式音频数据，连同头部信息与音视频bytes数据发送，支持多种采样率与音视频格式
|
||||
|
||||
#### 发送音频结束标志
|
||||
音频数据发送结束后，需要发送结束标志（需要用json序列化）：
|
||||
```text
|
||||
{"is_speaking": False}
|
||||
```
|
||||
|
||||
### 从服务端往客户端发数据
|
||||
#### 发送识别结果
|
||||
message为（采用json序列化）
|
||||
```text
|
||||
{"mode": "offline", "wav_name": "wav_name", "text": "asr ouputs", "is_final": True,"timestamp":"[[100,200], [200,500]]","stamp_sents":[]}
|
||||
```
|
||||
参数介绍：
|
||||
```text
|
||||
`mode`：`offline`，表示推理模式为离线文件转写
|
||||
`wav_name`：表示需要推理音频文件名
|
||||
`text`：表示语音识别输出文本
|
||||
`is_final`：表示识别结束，在 offline 模式下这个字段永远为 False，服务端 websocket 只会返回一次识别结果
|
||||
`timestamp`：如果AM为时间戳模型，会返回此字段，表示时间戳，格式为 "[[100,200], [200,500]]"(ms)
|
||||
`stamp_sents`：如果AM为时间戳模型，会返回此字段，表示句子级别时间戳，格式为 [{"text_seg":"正 是 因 为","punc":",","start":430,"end":1130,"ts_list":[[430,670],[670,810],[810,1030],[1030,1130]]}]
|
||||
```
|
||||
|
||||
## 实时语音识别
|
||||
### 系统架构图
|
||||
|
||||
<div align="left"><img src="images/2pass.jpg" width="600"/></div>
|
||||
|
||||
### 从客户端往服务端发送数据
|
||||
#### 消息格式
|
||||
配置参数与meta信息用json，音频数据采用bytes
|
||||
|
||||
#### 首次通信
|
||||
message为（需要用json序列化）：
|
||||
```text
|
||||
{"mode": "2pass", "wav_name": "wav_name", "is_speaking": True, "wav_format":"pcm", "chunk_size":[5,10,5], "hotwords":"{"阿里巴巴":20,"通义实验室":30}","itn":True}
|
||||
```
|
||||
参数介绍：
|
||||
```text
|
||||
`mode`：`offline`，表示推理模式为一句话识别；`online`，表示推理模式为实时语音识别；`2pass`：表示为实时语音识别，并且说话句尾采用离线模型进行纠错。
|
||||
`wav_name`：表示需要推理音频文件名
|
||||
`wav_format`：表示音视频文件后缀名，只支持pcm音频流
|
||||
`is_speaking`：表示断句尾点，例如，vad切割点，或者一条wav结束
|
||||
`chunk_size`：表示流式模型latency配置，`[5,10,5]`，表示当前音频为600ms，并且回看300ms，又看300ms。
|
||||
`audio_fs`：当输入音频为pcm数据是，需要加上音频采样率参数
|
||||
`hotwords`：如果使用热词，需要向服务端发送热词数据（字符串），格式为 "{"阿里巴巴":20,"通义实验室":30}"
|
||||
`itn`: 设置是否使用itn，默认True
|
||||
`svs_lang`: 设置SenseVoiceSmall模型语种，默认为“auto”
|
||||
`svs_itn`: 设置SenseVoiceSmall模型是否开启标点、ITN，默认为True
|
||||
```
|
||||
注：热词权重仅在fst热词服务下生效。
|
||||
|
||||
#### 发送音频数据
|
||||
直接将音频数据，移除头部信息后的bytes数据发送，支持音频采样率为8000（`message`中需要指定`audio_fs`为8000），16000
|
||||
#### 发送结束标志
|
||||
音频数据发送结束后，需要发送结束标志（需要用json序列化）：
|
||||
```text
|
||||
{"is_speaking": False}
|
||||
```
|
||||
### 从服务端往客户端发数据
|
||||
#### 发送识别结果
|
||||
message为（采用json序列化）
|
||||
```text
|
||||
{"mode": "2pass-online", "wav_name": "wav_name", "text": "asr ouputs", "is_final": True, "timestamp":"[[100,200], [200,500]]","stamp_sents":[]}
|
||||
```
|
||||
参数介绍：
|
||||
```text
|
||||
`mode`：表示推理模式，分为`2pass-online`，表示实时识别结果；`2pass-offline`，表示2遍修正识别结果
|
||||
`wav_name`：表示需要推理音频文件名
|
||||
`text`：表示语音识别输出文本
|
||||
`is_final`：表示识别结束
|
||||
`timestamp`：如果AM为时间戳模型，会返回此字段，表示时间戳，格式为 "[[100,200], [200,500]]"(ms)
|
||||
`stamp_sents`：如果AM为时间戳模型，会返回此字段，表示句子级别时间戳，格式为 [{"text_seg":"正 是 因 为","punc":",","start":430,"end":1130,"ts_list":[[430,670],[670,810],[810,1030],[1030,1130]]}]
|
||||
```
|
||||
Reference in New Issue
Block a user