Compare commits
10 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Nickolay Shmyrev
|
80e60b9118 | ||
|
Nickolay Shmyrev
|
e3a95a44bc | ||
|
Nickolay Shmyrev
|
86caf526f5 | ||
|
Nickolay Shmyrev
|
e09f32b4b4 | ||
|
Nickolay Shmyrev
|
3803ab345d | ||
|
Nickolay V. Shmyrev
|
889b43136f | ||
|
John Baber-Lucero
|
b517cf46af | ||
|
Nickolay Shmyrev
|
ffd810fe00 | ||
|
Nickolay V. Shmyrev
|
a1a0ed70a1 | ||
|
Nickolay Shmyrev
|
1554d9ede7 |
@@ -1,11 +1,19 @@
|
||||
[](https://travis-ci.com/alphacep/vosk-api)
|
||||
|
||||
Language bindings for Vosk and Kaldi to access speech recognition from various languages and on various platforms
|
||||
[РУС](README.ru.md)
|
||||
|
||||
* Python on Linux, Windows and RPi
|
||||
* Node
|
||||
* Android
|
||||
* iOS
|
||||
[中文](README.zh.md)
|
||||
|
||||
Vosk is a speech recognition toolkit. The best things in Vosk are:
|
||||
|
||||
1. Supports 9 languages - English, German, French, Spanish, Portuguese, Chinese, Russian, Turkish, Vietnamese. More to come.
|
||||
1. Works offline, even on lightweight devices - Raspberry Pi, Android, iOS
|
||||
1. Installs with simple `pip3 install vosk`
|
||||
1. Portable per-language models are only 50Mb each, but there are much bigger server models available.
|
||||
1. Provides streaming API for the best user experience (unlike popular speech-recognition python packages)
|
||||
1. There are bindings for different programming languages, too - java/csharp/javascript etc.
|
||||
1. Allows quick reconfiguration of vocabulary for best accuracy.
|
||||
1. Supports speaker identification beside simple speech recognition.
|
||||
|
||||
## Android build
|
||||
|
||||
@@ -81,6 +89,22 @@ cd python
|
||||
python3 setup.py install
|
||||
```
|
||||
|
||||
#### Running the example code with python
|
||||
|
||||
Run like this:
|
||||
|
||||
```
|
||||
cd vosk-api/python/example
|
||||
wget https://github.com/alphacep/kaldi-android-demo/releases/download/2020-01/alphacep-model-android-en-us-0.3.tar.gz
|
||||
tar xf alphacep-model-android-en-us-0.3.tar.gz
|
||||
mv alphacep-model-android-en-us-0.3 model-en
|
||||
python3 ./test_simple.py test.wav
|
||||
```
|
||||
|
||||
To run with your audio file make sure it has proper format - PCM 16khz 16bit mono, otherwise decoding will not work.
|
||||
|
||||
You can find other examples of using a microphone, decoding with a fixed small vocabulary or speaker identification setup in [python/example subfolder](https://github.com/alphacep/vosk-api/tree/master/python/example)
|
||||
|
||||
#### Java example API build
|
||||
|
||||
Or Java
|
||||
@@ -105,22 +129,6 @@ mv alphacep-model-android-en-us-0.3 model
|
||||
mono test.exe
|
||||
```
|
||||
|
||||
## Running the example code with python
|
||||
|
||||
Run like this:
|
||||
|
||||
```
|
||||
cd vosk-api/python/example
|
||||
wget https://github.com/alphacep/kaldi-android-demo/releases/download/2020-01/alphacep-model-android-en-us-0.3.tar.gz
|
||||
tar xf alphacep-model-android-en-us-0.3.tar.gz
|
||||
mv alphacep-model-android-en-us-0.3 model-en
|
||||
python3 ./test_simple.py test.wav
|
||||
```
|
||||
|
||||
To run with your audio file make sure it has proper format - PCM 16khz 16bit mono, otherwise decoding will not work.
|
||||
|
||||
You can find other examples of using a microphone, decoding with a fixed small vocabulary or speaker identification setup in [python/example subfolder](https://github.com/alphacep/vosk-api/tree/master/python/example)
|
||||
|
||||
## Models for different languages
|
||||
|
||||
For information about models see [the documentation on available models](https://github.com/alphacep/vosk-api/blob/master/doc/models.md).
|
||||
|
||||
+149
@@ -0,0 +1,149 @@
|
||||
[](https://travis-ci.com/alphacep/vosk-api)
|
||||
|
||||
[EN](README.md)
|
||||
|
||||
[中文](README.zh.md)
|
||||
|
||||
Библитека для распознавания речи "Воск". Преимущества библиотеки:
|
||||
|
||||
1. Поддерживает 9 языков - русский, английский, немецкий, французский, португальский, испанский, китайский, турецкий, вьетнамский. В скором времени будут добавлены и другие
|
||||
1. Работает без доступа к сети даже на мобильных устройствах - Raspberry Pi, Android, iOS
|
||||
1. Устанавливается с помощью простой команды `pip3 install vosk` без дополнительных шагов
|
||||
1. Модели для каждого языка занимают всего 50Мб, но есть и гораздо более точные большие модели для более точного распознавания
|
||||
1. Сделана для потоковой обработки звука, что позволяет реализовать мгновенную реакцию на команды
|
||||
1. Поддерживает несколько популярных языков программирования - Java, C#, Javascript
|
||||
1. Позволяет быстро настраивать словарь распознавания для улучшения точности распознавания
|
||||
1. Позволяет идентифицировать говорящего
|
||||
|
||||
## Сборка для Android
|
||||
|
||||
```
|
||||
cd android
|
||||
gradle build
|
||||
```
|
||||
|
||||
Сборка включает платформы x86, armv7, arm64
|
||||
|
||||
Для примера приложения, созданного с помощью библиотеки "Воск" смотрите [демо проект](https://github.com/alphacep/kaldi-android-demo).
|
||||
|
||||
## Сборка для iOS
|
||||
|
||||
Доступна позапросу. Напишите нам [contact@alphacephei.com](mailto:contact@alphacephei.com).
|
||||
|
||||
## Установка для работы Python из Pypi
|
||||
|
||||
Проще всего установить "Воск" с помощью pip. Собирать ничего не нужно. Мы поддерживаем платформы Linux, RPi и Windows. Сборка для OSX будет скоро доступна.
|
||||
|
||||
Для начала убедитесь, что используются достаточно новые версии pip и Python:
|
||||
|
||||
* Python версия >= 3.5
|
||||
* pip версия >= 19.0
|
||||
|
||||
Обновите Python и Pip если нужно, а затем установите "Воск" такой командой:
|
||||
|
||||
```
|
||||
pip3 install vosk
|
||||
```
|
||||
|
||||
Для использования "Воск" смотрите примеры ниже.
|
||||
|
||||
## Сервер для протоколов Websocket и GRPC
|
||||
|
||||
We also provide a websocket server and grpc server which can be used in telephony and other applications. With bigger models adapted for 8khz audio it provides more accuracy.
|
||||
|
||||
The server is installed with docker and can run with a single command:
|
||||
|
||||
```
|
||||
docker run -d -p 2700:2700 alphacep/kaldi-en:latest
|
||||
```
|
||||
|
||||
Смотрите проект https://github.com/alphacep/vosk-server
|
||||
|
||||
|
||||
## Сборка из исходников
|
||||
|
||||
Если нужно собрать проект из исходного кода, необходимо будет собрать
|
||||
Kaldi самостоятельно. Сборка досаточно простая и прямолинейная, но может
|
||||
быть непривычной для начинающих. Обращайте внимания на сообщения об ошибках
|
||||
в процессе сборки.
|
||||
|
||||
#### Сборка Kaldi для модулей на Python, Java, C#
|
||||
|
||||
```
|
||||
git clone https://github.com/kaldi-asr/kaldi
|
||||
cd kaldi/tools
|
||||
make
|
||||
```
|
||||
|
||||
установите все рекомандуемые пакеты и повторите `make` если потребуется.
|
||||
|
||||
```
|
||||
extras/install_openblas.sh
|
||||
cd ../src
|
||||
./configure --mathlib=OPENBLAS --shared --use-cuda=no
|
||||
make -j 10
|
||||
```
|
||||
|
||||
#### Сборка модуля на Python
|
||||
|
||||
После Kaldi можно собрать модуль Python
|
||||
|
||||
```
|
||||
export KALDI_ROOT=<KALDI_ROOT>
|
||||
cd python
|
||||
python3 setup.py install
|
||||
```
|
||||
|
||||
#### Запуск примера для Python
|
||||
|
||||
Выполните следующие команды:
|
||||
|
||||
```
|
||||
cd vosk-api/python/example
|
||||
wget https://github.com/alphacep/kaldi-android-demo/releases/download/2020-01/alphacep-model-android-en-us-0.3.tar.gz
|
||||
tar xf alphacep-model-android-en-us-0.3.tar.gz
|
||||
mv alphacep-model-android-en-us-0.3 model-en
|
||||
python3 ./test_simple.py test.wav
|
||||
```
|
||||
|
||||
Для того, чтобы распознавать другой файл, переведите его в нужный формат - PCM 16кГц 16бит 1канал. Это можно сделать с помощью ffmpeg.
|
||||
|
||||
Другие примеры, в том числе использования микрофона, распознавание с небольшим словарём и распознавание говорящего можно найти в [каталоге python/example](https://github.com/alphacep/vosk-api/tree/master/python/example)
|
||||
|
||||
#### Сборка для Java
|
||||
|
||||
Перейдите в каталог Java и запустите сборку
|
||||
|
||||
```
|
||||
cd java && KALDI_ROOT=<KALDI_ROOT> make
|
||||
wget https://github.com/alphacep/kaldi-android-demo/releases/download/2020-01/alphacep-model-android-en-us-0.3.tar.gz
|
||||
tar xf alphacep-model-android-en-us-0.3.tar.gz
|
||||
mv alphacep-model-android-en-us-0.3 model
|
||||
make run
|
||||
```
|
||||
|
||||
#### Сборка для C#
|
||||
|
||||
Для сборки в среде Mono.
|
||||
|
||||
```
|
||||
cd csharp && KALDI_ROOT=<KALDI_ROOT> make
|
||||
wget https://github.com/alphacep/kaldi-android-demo/releases/download/2020-01/alphacep-model-android-en-us-0.3.tar.gz
|
||||
tar xf alphacep-model-android-en-us-0.3.tar.gz
|
||||
mv alphacep-model-android-en-us-0.3 model
|
||||
mono test.exe
|
||||
```
|
||||
|
||||
.NET тоже должен работать, хотя мы не пробовали.
|
||||
|
||||
## Модели для разных языков
|
||||
|
||||
По информации о моделях смотрите соответствующую [страницу документации](https://github.com/alphacep/vosk-api/blob/master/doc/models.md).
|
||||
|
||||
## Contact Us
|
||||
|
||||
Если возникли вопросы, свяжитесь с нами:
|
||||
|
||||
* Создайте проблему тут на github
|
||||
* Напишите нам по почте [contact@alphacephei.com](mailto:contact@alphacephei.com)
|
||||
* Заходите в нашу группу в Телеграмме [@speech_recognition_ru](https://t.me/speech_recognition_ru)
|
||||
@@ -0,0 +1,12 @@
|
||||
[](https://travis-ci.com/alphacep/vosk-api)
|
||||
|
||||
Vosk是言语识别工具包。Vosk最好的事情是：
|
||||
|
||||
1. 支持九种语言 - 中文, 英语，德语，法语，西班牙语，葡萄牙语，俄语，土耳其语，越南语
|
||||
1. 移动设备上脱机工作-Raspberry Pi，Android，iOS
|
||||
1. 使用简单的 pip3 install vosk 安装
|
||||
1. 每种语言的手提式模型只有是50Mb, 但还有更大的服务器模型可用
|
||||
1. 提供流媒体API，以提供最佳用户体验（与流行的语音识别python包不同）
|
||||
1. 还有用于不同编程语言的包装器-java / csharp / javascript等
|
||||
1. 可以快速重新配置词汇以实现最佳准确性
|
||||
1. 支持说话人识别
|
||||
@@ -149,7 +149,7 @@ public class SpeechRecognizer {
|
||||
boolean result = stopRecognizerThread();
|
||||
if (result) {
|
||||
Log.i(TAG, "Stop recognition");
|
||||
mainHandler.post(new ResultEvent(recognizer.FinalResult(), true));
|
||||
mainHandler.post(new ResultEvent(recognizer.Result(), true));
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -162,6 +162,7 @@ public class SpeechRecognizer {
|
||||
*/
|
||||
public boolean cancel() {
|
||||
boolean result = stopRecognizerThread();
|
||||
recognizer.Result(); // Reset recognizer state
|
||||
if (result) {
|
||||
Log.i(TAG, "Cancel recognition");
|
||||
}
|
||||
|
||||
@@ -11,6 +11,7 @@ KALDI_LIBS = \
|
||||
${KALDI_ROOT}/src/tree/kaldi-tree.a \
|
||||
${KALDI_ROOT}/src/feat/kaldi-feat.a \
|
||||
${KALDI_ROOT}/src/lat/kaldi-lat.a \
|
||||
${KALDI_ROOT}/src/lm/kaldi-lm.a \
|
||||
${KALDI_ROOT}/src/hmm/kaldi-hmm.a \
|
||||
${KALDI_ROOT}/src/transform/kaldi-transform.a \
|
||||
${KALDI_ROOT}/src/cudamatrix/kaldi-cudamatrix.a \
|
||||
|
||||
+9
-1
@@ -22,7 +22,7 @@ To add a new model here create an issue on Github.
|
||||
|
||||
| Model | Size | Accuracy | Notes |
|
||||
|-----------------------------------------------------------------------------------------------------------|-------|------------|----------------------------------------------------------------------------------------------|
|
||||
| [kaldi-ru-0.9.tar.gz](http://alphacephei.com/kaldi/kaldi-ru-0.9.tar.gz) | 2.5G | TBD | Big narrowband Russian model for server processing |
|
||||
| [kaldi-ru-0.10.tar.gz](http://alphacephei.com/kaldi/kaldi-ru-0.10.tar.gz) | 2.5G | TBD | Big narrowband Russian model for server processing |
|
||||
| [alphacep-model-android-ru-0.3](http://alphacephei.com/kaldi/alphacep-model-android-ru-0.3.tar.gz) | 39M | TBD | Lightweight wideband model for Android and RPi |
|
||||
|
||||
### French
|
||||
@@ -60,6 +60,14 @@ https://github.com/opensource-spraakherkenning-nl/Kaldi_NL
|
||||
|-----------------------------------------------------------------------------------------------------------|-------|------------|----------------------------------------------------------------------------------------------|
|
||||
| [kaldi-el-gr-0.6.tar.gz](http://alphacephei.com/kaldi/kaldi-el-gr-0.6.tar.gz) | 1.1G | TBD | Big narrowband Greek model for server processing, not extremely accurate though |
|
||||
|
||||
|
||||
### Turkish
|
||||
|
||||
| Model | Size | Accuracy | Notes |
|
||||
|-----------------------------------------------------------------------------------------------------------|-------|------------|----------------------------------------------------------------------------------------------|
|
||||
| [alphacep-model-android-tr-0.3](http://alphacephei.com/kaldi/alphacep-model-android-tr-0.3.tar.gz) | 35M | TBD | Lightweight wideband model for Android and RPi |
|
||||
|
||||
|
||||
### Vietnamese
|
||||
|
||||
| Model | Size | Accuracy | Notes |
|
||||
|
||||
@@ -11,6 +11,7 @@ KALDI_LIBS = \
|
||||
${KALDI_ROOT}/src/tree/kaldi-tree.a \
|
||||
${KALDI_ROOT}/src/feat/kaldi-feat.a \
|
||||
${KALDI_ROOT}/src/lat/kaldi-lat.a \
|
||||
${KALDI_ROOT}/src/lm/kaldi-lm.a \
|
||||
${KALDI_ROOT}/src/hmm/kaldi-hmm.a \
|
||||
${KALDI_ROOT}/src/transform/kaldi-transform.a \
|
||||
${KALDI_ROOT}/src/cudamatrix/kaldi-cudamatrix.a \
|
||||
|
||||
@@ -19,7 +19,7 @@ public class DecoderTest {
|
||||
|
||||
public static void main(String args[]) throws IOException {
|
||||
FileInputStream ais = new FileInputStream(new File("../python/example/test.wav"));
|
||||
Model model = new Model("model-en");
|
||||
Model model = new Model("model");
|
||||
SpkModel spkModel = new SpkModel("model-spk");
|
||||
KaldiRecognizer rec = new KaldiRecognizer(model, spkModel, 16000.0f);
|
||||
|
||||
|
||||
@@ -3,18 +3,18 @@ project(vosk)
|
||||
|
||||
set(TOP_SRCDIR "${CMAKE_SOURCE_DIR}/..")
|
||||
if("x$ENV{WHEEL_FLAGS}" STREQUAL "x")
|
||||
find_package (Python COMPONENTS Interpreter Development)
|
||||
find_package (Python3 COMPONENTS Interpreter Development)
|
||||
else()
|
||||
# docker case
|
||||
set(Python_INCLUDE_DIRS "")
|
||||
set(Python3_INCLUDE_DIRS "")
|
||||
set(TOP_SRCDIR "/io")
|
||||
endif()
|
||||
|
||||
set(KALDI_ROOT "$ENV{KALDI_ROOT}")
|
||||
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -O3 -DFST_NO_DYNAMIC_LINKING")
|
||||
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DFST_NO_DYNAMIC_LINKING")
|
||||
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} $ENV{WHEEL_FLAGS}")
|
||||
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CMAKE_C_FLAGS} -std=c++11")
|
||||
include_directories("${TOP_SRCDIR}/src" "${KALDI_ROOT}/src" "${KALDI_ROOT}/tools/openfst/include" ${Python_INCLUDE_DIRS})
|
||||
include_directories("${TOP_SRCDIR}/src" "${KALDI_ROOT}/src" "${KALDI_ROOT}/tools/openfst/include" ${Python3_INCLUDE_DIRS})
|
||||
|
||||
find_package(SWIG REQUIRED)
|
||||
include(${SWIG_USE_FILE})
|
||||
@@ -35,6 +35,7 @@ swig_link_libraries(vosk
|
||||
${KALDI_ROOT}/src/tree/kaldi-tree.a
|
||||
${KALDI_ROOT}/src/feat/kaldi-feat.a
|
||||
${KALDI_ROOT}/src/lat/kaldi-lat.a
|
||||
${KALDI_ROOT}/src/lm/kaldi-lm.a
|
||||
${KALDI_ROOT}/src/hmm/kaldi-hmm.a
|
||||
${KALDI_ROOT}/src/transform/kaldi-transform.a
|
||||
${KALDI_ROOT}/src/cudamatrix/kaldi-cudamatrix.a
|
||||
|
||||
@@ -4,7 +4,7 @@ from vosk import Model, KaldiRecognizer
|
||||
import sys
|
||||
import json
|
||||
|
||||
model = Model("model-en")
|
||||
model = Model("model")
|
||||
rec = KaldiRecognizer(model, 8000)
|
||||
|
||||
res = json.loads(rec.FinalResult())
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
from vosk import Model, KaldiRecognizer
|
||||
import os
|
||||
|
||||
if not os.path.exists("model-en"):
|
||||
print ("Please download the model from https://github.com/alphacep/kaldi-android-demo/releases and unpack as 'model-en' in the current folder.")
|
||||
if not os.path.exists("model"):
|
||||
print ("Please download the model from https://github.com/alphacep/kaldi-android-demo/releases and unpack as 'model' in the current folder.")
|
||||
exit (1)
|
||||
|
||||
import pyaudio
|
||||
@@ -13,7 +13,7 @@ p = pyaudio.PyAudio()
|
||||
stream = p.open(format=pyaudio.paInt16, channels=1, rate=16000, input=True, frames_per_buffer=8000)
|
||||
stream.start_stream()
|
||||
|
||||
model = Model("model-en")
|
||||
model = Model("model")
|
||||
rec = KaldiRecognizer(model, 16000)
|
||||
|
||||
while True:
|
||||
|
||||
@@ -5,8 +5,8 @@ import sys
|
||||
import os
|
||||
import wave
|
||||
|
||||
if not os.path.exists("model-en"):
|
||||
print ("Please download the model from https://github.com/alphacep/kaldi-android-demo/releases and unpack as 'model-en' in the current folder.")
|
||||
if not os.path.exists("model"):
|
||||
print ("Please download the model from https://github.com/alphacep/kaldi-android-demo/releases and unpack as 'model' in the current folder.")
|
||||
exit (1)
|
||||
|
||||
wf = wave.open(sys.argv[1], "rb")
|
||||
@@ -14,7 +14,7 @@ if wf.getnchannels() != 1 or wf.getsampwidth() != 2 or wf.getcomptype() != "NONE
|
||||
print ("Audio file must be WAV format mono PCM.")
|
||||
exit (1)
|
||||
|
||||
model = Model("model-en")
|
||||
model = Model("model")
|
||||
rec = KaldiRecognizer(model, wf.getframerate())
|
||||
|
||||
while True:
|
||||
|
||||
@@ -7,7 +7,7 @@ import json
|
||||
import os
|
||||
import numpy as np
|
||||
|
||||
model_path = "model-en"
|
||||
model_path = "model"
|
||||
spk_model_path = "model-spk"
|
||||
|
||||
if not os.path.exists(model_path):
|
||||
|
||||
@@ -5,12 +5,12 @@ import sys
|
||||
import json
|
||||
import os
|
||||
|
||||
if not os.path.exists("model-en"):
|
||||
print ("Please download the model from https://github.com/alphacep/kaldi-android-demo/releases and unpack as 'model-en' in the current folder.")
|
||||
if not os.path.exists("model"):
|
||||
print ("Please download the model from https://github.com/alphacep/kaldi-android-demo/releases and unpack as 'model' in the current folder.")
|
||||
exit (1)
|
||||
|
||||
|
||||
model = Model("model-en")
|
||||
model = Model("model")
|
||||
|
||||
# Large vocabulary free form recognition
|
||||
rec = KaldiRecognizer(model, 16000)
|
||||
|
||||
@@ -5,8 +5,8 @@ import sys
|
||||
import os
|
||||
import wave
|
||||
|
||||
if not os.path.exists("model-en"):
|
||||
print ("Please download the model from https://github.com/alphacep/kaldi-android-demo/releases and unpack as 'model-en' in the current folder.")
|
||||
if not os.path.exists("model"):
|
||||
print ("Please download the model from https://github.com/alphacep/kaldi-android-demo/releases and unpack as 'model' in the current folder.")
|
||||
exit (1)
|
||||
|
||||
wf = wave.open(sys.argv[1], "rb")
|
||||
@@ -14,7 +14,7 @@ if wf.getnchannels() != 1 or wf.getsampwidth() != 2 or wf.getcomptype() != "NONE
|
||||
print ("Audio file must be WAV format mono PCM.")
|
||||
exit (1)
|
||||
|
||||
model = Model("model-en")
|
||||
model = Model("model")
|
||||
# You can also specify the possible word list
|
||||
rec = KaldiRecognizer(model, wf.getframerate(), "zero oh one two three four five six seven eight nine")
|
||||
|
||||
|
||||
+1
-1
@@ -7,7 +7,7 @@ with open("README.md", "r") as fh:
|
||||
|
||||
setuptools.setup(
|
||||
name="vosk", # Replace with your own username
|
||||
version="0.3.4",
|
||||
version="0.3.6",
|
||||
author="Alpha Cephei Inc",
|
||||
author_email="contact@alphacephei.com",
|
||||
description="API for Kaldi and Vosk",
|
||||
|
||||
Executable
+27
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/python3
|
||||
|
||||
from vosk import Model, KaldiRecognizer
|
||||
import sys
|
||||
import os
|
||||
import wave
|
||||
import json
|
||||
|
||||
model = Model("model")
|
||||
|
||||
for line in open(sys.argv[1]):
|
||||
uid, fn = line.split()
|
||||
wf = wave.open(fn, "rb")
|
||||
rec = KaldiRecognizer(model, wf.getframerate())
|
||||
|
||||
text = ""
|
||||
while True:
|
||||
data = wf.readframes(1000)
|
||||
if len(data) == 0:
|
||||
break
|
||||
if rec.AcceptWaveform(data):
|
||||
jres = json.loads(rec.Result())
|
||||
text = text + " " + jres['text']
|
||||
jres = json.loads(rec.FinalResult())
|
||||
text = text + " " + jres['text']
|
||||
print (uid + text)
|
||||
|
||||
+102
-52
@@ -20,40 +20,46 @@
|
||||
using namespace fst;
|
||||
using namespace kaldi::nnet3;
|
||||
|
||||
KaldiRecognizer::KaldiRecognizer(Model &model, float sample_frequency) : model_(model), spk_model_(0), sample_frequency_(sample_frequency) {
|
||||
KaldiRecognizer::KaldiRecognizer(Model *model, float sample_frequency) : model_(model), spk_model_(0), sample_frequency_(sample_frequency) {
|
||||
|
||||
feature_pipeline_ = new kaldi::OnlineNnet2FeaturePipeline (model_.feature_info_);
|
||||
silence_weighting_ = new kaldi::OnlineSilenceWeighting(*model_.trans_model_, model_.feature_info_.silence_weighting_config, 3);
|
||||
model_->Ref();
|
||||
|
||||
feature_pipeline_ = new kaldi::OnlineNnet2FeaturePipeline (model_->feature_info_);
|
||||
silence_weighting_ = new kaldi::OnlineSilenceWeighting(*model_->trans_model_, model_->feature_info_.silence_weighting_config, 3);
|
||||
|
||||
g_fst_ = NULL;
|
||||
decode_fst_ = NULL;
|
||||
|
||||
if (!model_.hclg_fst_) {
|
||||
if (model_.hcl_fst_ && model_.g_fst_) {
|
||||
decode_fst_ = LookaheadComposeFst(*model_.hcl_fst_, *model_.g_fst_, model_.disambig_);
|
||||
if (!model_->hclg_fst_) {
|
||||
if (model_->hcl_fst_ && model_->g_fst_) {
|
||||
decode_fst_ = LookaheadComposeFst(*model_->hcl_fst_, *model_->g_fst_, model_->disambig_);
|
||||
} else {
|
||||
KALDI_ERR << "Can't create decoding graph";
|
||||
}
|
||||
}
|
||||
|
||||
decoder_ = new kaldi::SingleUtteranceNnet3Decoder(model_.nnet3_decoding_config_,
|
||||
*model_.trans_model_,
|
||||
*model_.decodable_info_,
|
||||
model_.hclg_fst_ ? *model.hclg_fst_ : *decode_fst_,
|
||||
decoder_ = new kaldi::SingleUtteranceNnet3Decoder(model_->nnet3_decoding_config_,
|
||||
*model_->trans_model_,
|
||||
*model_->decodable_info_,
|
||||
model_->hclg_fst_ ? *model_->hclg_fst_ : *decode_fst_,
|
||||
feature_pipeline_);
|
||||
|
||||
frame_offset_ = 0;
|
||||
input_finalized_ = false;
|
||||
spk_feature_ = NULL;
|
||||
|
||||
InitRescoring();
|
||||
}
|
||||
|
||||
KaldiRecognizer::KaldiRecognizer(Model &model, float sample_frequency, char const *grammar) : model_(model), spk_model_(0), sample_frequency_(sample_frequency)
|
||||
KaldiRecognizer::KaldiRecognizer(Model *model, float sample_frequency, char const *grammar) : model_(model), spk_model_(0), sample_frequency_(sample_frequency)
|
||||
{
|
||||
feature_pipeline_ = new kaldi::OnlineNnet2FeaturePipeline (model_.feature_info_);
|
||||
silence_weighting_ = new kaldi::OnlineSilenceWeighting(*model_.trans_model_, model_.feature_info_.silence_weighting_config, 3);
|
||||
model_->Ref();
|
||||
|
||||
feature_pipeline_ = new kaldi::OnlineNnet2FeaturePipeline (model_->feature_info_);
|
||||
silence_weighting_ = new kaldi::OnlineSilenceWeighting(*model_->trans_model_, model_->feature_info_.silence_weighting_config, 3);
|
||||
|
||||
g_fst_ = new StdVectorFst();
|
||||
if (model_.hcl_fst_) {
|
||||
if (model_->hcl_fst_) {
|
||||
g_fst_->AddState();
|
||||
g_fst_->SetStart(0);
|
||||
g_fst_->AddState();
|
||||
@@ -61,58 +67,65 @@ KaldiRecognizer::KaldiRecognizer(Model &model, float sample_frequency, char cons
|
||||
g_fst_->AddArc(1, StdArc(0, 0, fst::TropicalWeight::One(), 0));
|
||||
|
||||
// Create simple word loop FST
|
||||
std::stringstream ss(grammar);
|
||||
std::string token;
|
||||
stringstream ss(grammar);
|
||||
string token;
|
||||
|
||||
while (std::getline(ss, token, ' ')) {
|
||||
int32 id = model_.word_syms_->Find(token);
|
||||
while (getline(ss, token, ' ')) {
|
||||
int32 id = model_->word_syms_->Find(token);
|
||||
g_fst_->AddArc(0, StdArc(id, id, fst::TropicalWeight::One(), 1));
|
||||
}
|
||||
ArcSort(g_fst_, ILabelCompare<StdArc>());
|
||||
|
||||
decode_fst_ = LookaheadComposeFst(*model_.hcl_fst_, *g_fst_, model_.disambig_);
|
||||
decode_fst_ = LookaheadComposeFst(*model_->hcl_fst_, *g_fst_, model_->disambig_);
|
||||
} else {
|
||||
decode_fst_ = NULL;
|
||||
KALDI_ERR << "Can't create decoding graph";
|
||||
}
|
||||
|
||||
decoder_ = new kaldi::SingleUtteranceNnet3Decoder(model_.nnet3_decoding_config_,
|
||||
*model_.trans_model_,
|
||||
*model_.decodable_info_,
|
||||
model_.hclg_fst_ ? *model.hclg_fst_ : *decode_fst_,
|
||||
decoder_ = new kaldi::SingleUtteranceNnet3Decoder(model_->nnet3_decoding_config_,
|
||||
*model_->trans_model_,
|
||||
*model_->decodable_info_,
|
||||
model_->hclg_fst_ ? *model_->hclg_fst_ : *decode_fst_,
|
||||
feature_pipeline_);
|
||||
|
||||
frame_offset_ = 0;
|
||||
input_finalized_ = false;
|
||||
spk_feature_ = NULL;
|
||||
|
||||
InitRescoring();
|
||||
}
|
||||
|
||||
KaldiRecognizer::KaldiRecognizer(Model *model, SpkModel *spk_model, float sample_frequency) : model_(model), spk_model_(spk_model), sample_frequency_(sample_frequency) {
|
||||
|
||||
KaldiRecognizer::KaldiRecognizer(Model &model, SpkModel *spk_model, float sample_frequency) : model_(model), spk_model_(spk_model), sample_frequency_(sample_frequency) {
|
||||
feature_pipeline_ = new kaldi::OnlineNnet2FeaturePipeline (model_.feature_info_);
|
||||
silence_weighting_ = new kaldi::OnlineSilenceWeighting(*model_.trans_model_, model_.feature_info_.silence_weighting_config, 3);
|
||||
model_->Ref();
|
||||
spk_model->Ref();
|
||||
|
||||
feature_pipeline_ = new kaldi::OnlineNnet2FeaturePipeline (model_->feature_info_);
|
||||
silence_weighting_ = new kaldi::OnlineSilenceWeighting(*model_->trans_model_, model_->feature_info_.silence_weighting_config, 3);
|
||||
|
||||
decode_fst_ = NULL;
|
||||
g_fst_ = NULL;
|
||||
|
||||
if (!model_.hclg_fst_) {
|
||||
if (model_.hcl_fst_ && model_.g_fst_) {
|
||||
decode_fst_ = LookaheadComposeFst(*model_.hcl_fst_, *model_.g_fst_, model_.disambig_);
|
||||
if (!model_->hclg_fst_) {
|
||||
if (model_->hcl_fst_ && model_->g_fst_) {
|
||||
decode_fst_ = LookaheadComposeFst(*model_->hcl_fst_, *model_->g_fst_, model_->disambig_);
|
||||
} else {
|
||||
KALDI_ERR << "Can't create decoding graph";
|
||||
}
|
||||
}
|
||||
|
||||
decoder_ = new kaldi::SingleUtteranceNnet3Decoder(model_.nnet3_decoding_config_,
|
||||
*model_.trans_model_,
|
||||
*model_.decodable_info_,
|
||||
model_.hclg_fst_ ? *model.hclg_fst_ : *decode_fst_,
|
||||
decoder_ = new kaldi::SingleUtteranceNnet3Decoder(model_->nnet3_decoding_config_,
|
||||
*model_->trans_model_,
|
||||
*model_->decodable_info_,
|
||||
model_->hclg_fst_ ? *model_->hclg_fst_ : *decode_fst_,
|
||||
feature_pipeline_);
|
||||
|
||||
frame_offset_ = 0;
|
||||
input_finalized_ = false;
|
||||
|
||||
spk_feature_ = new OnlineMfcc(spk_model_->spkvector_mfcc_opts);
|
||||
|
||||
InitRescoring();
|
||||
}
|
||||
|
||||
KaldiRecognizer::~KaldiRecognizer() {
|
||||
@@ -122,12 +135,29 @@ KaldiRecognizer::~KaldiRecognizer() {
|
||||
delete g_fst_;
|
||||
delete decode_fst_;
|
||||
delete spk_feature_;
|
||||
delete lm_fst_;
|
||||
|
||||
model_->Unref();
|
||||
if (spk_model_)
|
||||
spk_model_->Unref();
|
||||
}
|
||||
|
||||
void KaldiRecognizer::InitRescoring()
|
||||
{
|
||||
if (model_->std_lm_fst_) {
|
||||
fst::CacheOptions cache_opts(true, 50000);
|
||||
fst::MapFstOptions mapfst_opts(cache_opts);
|
||||
fst::StdToLatticeMapper<kaldi::BaseFloat> mapper;
|
||||
lm_fst_ = new fst::MapFst<fst::StdArc, kaldi::LatticeArc, fst::StdToLatticeMapper<kaldi::BaseFloat> >(*model_->std_lm_fst_, mapper, mapfst_opts);
|
||||
} else {
|
||||
lm_fst_ = NULL;
|
||||
}
|
||||
}
|
||||
|
||||
void KaldiRecognizer::CleanUp()
|
||||
{
|
||||
delete silence_weighting_;
|
||||
silence_weighting_ = new kaldi::OnlineSilenceWeighting(*model_.trans_model_, model_.feature_info_.silence_weighting_config, 3);
|
||||
silence_weighting_ = new kaldi::OnlineSilenceWeighting(*model_->trans_model_, model_->feature_info_.silence_weighting_config, 3);
|
||||
|
||||
frame_offset_ += decoder_->NumFramesDecoded();
|
||||
decoder_->InitDecoding(frame_offset_);
|
||||
@@ -137,7 +167,7 @@ void KaldiRecognizer::UpdateSilenceWeights()
|
||||
{
|
||||
if (silence_weighting_->Active() && feature_pipeline_->NumFramesReady() > 0 &&
|
||||
feature_pipeline_->IvectorFeature() != NULL) {
|
||||
std::vector<std::pair<int32, BaseFloat> > delta_weights;
|
||||
vector<pair<int32, BaseFloat> > delta_weights;
|
||||
silence_weighting_->ComputeCurrentTraceback(decoder_->Decoder());
|
||||
silence_weighting_->GetDeltaWeights(feature_pipeline_->NumFramesReady(),
|
||||
frame_offset_ * 3,
|
||||
@@ -188,14 +218,13 @@ bool KaldiRecognizer::AcceptWaveform(Vector<BaseFloat> &wdata)
|
||||
spk_feature_->AcceptWaveform(sample_frequency_, wdata);
|
||||
}
|
||||
|
||||
if (decoder_->EndpointDetected(model_.endpoint_config_)) {
|
||||
if (decoder_->EndpointDetected(model_->endpoint_config_)) {
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
|
||||
// Computes an xvector from a chunk of speech features.
|
||||
static void RunNnetComputation(const MatrixBase<BaseFloat> &features,
|
||||
const nnet3::Nnet &nnet, nnet3::CachingOptimizingCompiler *compiler,
|
||||
@@ -212,7 +241,7 @@ static void RunNnetComputation(const MatrixBase<BaseFloat> &features,
|
||||
output_spec.indexes.resize(1);
|
||||
request.outputs.resize(1);
|
||||
request.outputs[0].Swap(&output_spec);
|
||||
std::shared_ptr<const nnet3::NnetComputation> computation = compiler->Compile(request);
|
||||
shared_ptr<const nnet3::NnetComputation> computation = compiler->Compile(request);
|
||||
nnet3::Nnet *nnet_to_update = NULL; // we're not doing any update.
|
||||
nnet3::NnetComputer computer(nnet3::NnetComputeOptions(), *computation,
|
||||
nnet, nnet_to_update);
|
||||
@@ -225,7 +254,6 @@ static void RunNnetComputation(const MatrixBase<BaseFloat> &features,
|
||||
xvector->CopyFromVec(cu_output.Row(0));
|
||||
}
|
||||
|
||||
|
||||
void KaldiRecognizer::GetSpkVector(Vector<BaseFloat> &xvector)
|
||||
{
|
||||
int num_frames = spk_feature_->NumFramesReady() - frame_offset_ * 3;
|
||||
@@ -246,7 +274,6 @@ void KaldiRecognizer::GetSpkVector(Vector<BaseFloat> &xvector)
|
||||
RunNnetComputation(features, spk_model_->speaker_nnet, &compiler, &xvector);
|
||||
}
|
||||
|
||||
|
||||
const char* KaldiRecognizer::Result()
|
||||
{
|
||||
|
||||
@@ -262,30 +289,53 @@ const char* KaldiRecognizer::Result()
|
||||
|
||||
kaldi::CompactLattice clat;
|
||||
decoder_->GetLattice(true, &clat);
|
||||
fst::ScaleLattice(fst::LatticeScale(8.0, 10.0), &clat);
|
||||
|
||||
if (model_->std_lm_fst_) {
|
||||
Lattice lat1;
|
||||
|
||||
ConvertLattice(clat, &lat1);
|
||||
fst::ScaleLattice(fst::GraphLatticeScale(-1.0), &lat1);
|
||||
fst::ArcSort(&lat1, fst::OLabelCompare<kaldi::LatticeArc>());
|
||||
kaldi::Lattice composed_lat;
|
||||
fst::Compose(lat1, *lm_fst_, &composed_lat);
|
||||
fst::Invert(&composed_lat);
|
||||
kaldi::CompactLattice determinized_lat;
|
||||
DeterminizeLattice(composed_lat, &determinized_lat);
|
||||
fst::ScaleLattice(fst::GraphLatticeScale(-1), &determinized_lat);
|
||||
fst::ArcSort(&determinized_lat, fst::OLabelCompare<kaldi::CompactLatticeArc>());
|
||||
|
||||
kaldi::ConstArpaLmDeterministicFst const_arpa_fst(model_->const_arpa_);
|
||||
kaldi::CompactLattice composed_clat;
|
||||
kaldi::ComposeCompactLatticeDeterministic(determinized_lat, &const_arpa_fst, &composed_clat);
|
||||
kaldi::Lattice composed_lat1;
|
||||
ConvertLattice(composed_clat, &composed_lat1);
|
||||
fst::Invert(&composed_lat1);
|
||||
DeterminizeLattice(composed_lat1, &clat);
|
||||
}
|
||||
|
||||
fst::ScaleLattice(fst::LatticeScale(9.0, 10.0), &clat);
|
||||
CompactLattice aligned_lat;
|
||||
if (model_.winfo_) {
|
||||
WordAlignLattice(clat, *model_.trans_model_, *model_.winfo_, 0, &aligned_lat);
|
||||
if (model_->winfo_) {
|
||||
WordAlignLattice(clat, *model_->trans_model_, *model_->winfo_, 0, &aligned_lat);
|
||||
} else {
|
||||
aligned_lat = clat;
|
||||
}
|
||||
|
||||
MinimumBayesRisk mbr(aligned_lat);
|
||||
const std::vector<BaseFloat> &conf = mbr.GetOneBestConfidences();
|
||||
const std::vector<int32> &words = mbr.GetOneBest();
|
||||
const std::vector<std::pair<BaseFloat, BaseFloat> > × =
|
||||
const vector<BaseFloat> &conf = mbr.GetOneBestConfidences();
|
||||
const vector<int32> &words = mbr.GetOneBest();
|
||||
const vector<pair<BaseFloat, BaseFloat> > × =
|
||||
mbr.GetOneBestTimes();
|
||||
|
||||
int size = words.size();
|
||||
|
||||
json::JSON obj;
|
||||
std::stringstream text;
|
||||
stringstream text;
|
||||
|
||||
// Create JSON object
|
||||
for (int i = 0; i < size; i++) {
|
||||
json::JSON word;
|
||||
word["word"] = model_.word_syms_->Find(words[i]);
|
||||
word["word"] = model_->word_syms_->Find(words[i]);
|
||||
word["start"] = (frame_offset_ + times[i].first) * 0.03;
|
||||
word["end"] = (frame_offset_ + times[i].second) * 0.03;
|
||||
word["conf"] = conf[i];
|
||||
@@ -294,7 +344,7 @@ const char* KaldiRecognizer::Result()
|
||||
if (i) {
|
||||
text << " ";
|
||||
}
|
||||
text << model_.word_syms_->Find(words[i]);
|
||||
text << model_->word_syms_->Find(words[i]);
|
||||
}
|
||||
obj["text"] = text.str();
|
||||
|
||||
@@ -321,16 +371,16 @@ const char* KaldiRecognizer::PartialResult()
|
||||
|
||||
kaldi::Lattice lat;
|
||||
decoder_->GetBestPath(false, &lat);
|
||||
std::vector<kaldi::int32> alignment, words;
|
||||
vector<kaldi::int32> alignment, words;
|
||||
LatticeWeight weight;
|
||||
GetLinearSymbolSequence(lat, &alignment, &words, &weight);
|
||||
|
||||
std::ostringstream text;
|
||||
ostringstream text;
|
||||
for (size_t i = 0; i < words.size(); i++) {
|
||||
if (i) {
|
||||
text << " ";
|
||||
}
|
||||
text << model_.word_syms_->Find(words[i]);
|
||||
text << model_->word_syms_->Find(words[i]);
|
||||
}
|
||||
res["partial"] = text.str();
|
||||
|
||||
|
||||
@@ -31,9 +31,9 @@ using namespace kaldi;
|
||||
|
||||
class KaldiRecognizer {
|
||||
public:
|
||||
KaldiRecognizer(Model &model, float sample_frequency);
|
||||
KaldiRecognizer(Model &model, SpkModel *spk_model, float sample_frequency);
|
||||
KaldiRecognizer(Model &model, float sample_frequency, char const *grammar);
|
||||
KaldiRecognizer(Model *model, float sample_frequency);
|
||||
KaldiRecognizer(Model *model, SpkModel *spk_model, float sample_frequency);
|
||||
KaldiRecognizer(Model *model, float sample_frequency, char const *grammar);
|
||||
~KaldiRecognizer();
|
||||
bool AcceptWaveform(const char *data, int len);
|
||||
bool AcceptWaveform(const short *sdata, int len);
|
||||
@@ -43,12 +43,13 @@ class KaldiRecognizer {
|
||||
const char* PartialResult();
|
||||
|
||||
private:
|
||||
void InitRescoring();
|
||||
void CleanUp();
|
||||
void UpdateSilenceWeights();
|
||||
bool AcceptWaveform(Vector<BaseFloat> &wdata);
|
||||
void GetSpkVector(Vector<BaseFloat> &xvector);
|
||||
|
||||
Model &model_;
|
||||
Model *model_;
|
||||
SingleUtteranceNnet3Decoder *decoder_;
|
||||
fst::LookaheadFst<fst::StdArc, int32> *decode_fst_;
|
||||
fst::StdVectorFst *g_fst_; // dynamically constructed grammar
|
||||
@@ -58,8 +59,10 @@ class KaldiRecognizer {
|
||||
SpkModel *spk_model_;
|
||||
OnlineBaseFeature *spk_feature_;
|
||||
|
||||
fst::MapFst<fst::StdArc, kaldi::LatticeArc, fst::StdToLatticeMapper<kaldi::BaseFloat> > *lm_fst_;
|
||||
|
||||
float sample_frequency_;
|
||||
int32 frame_offset_;
|
||||
bool input_finalized_;
|
||||
std::string last_result_;
|
||||
string last_result_;
|
||||
};
|
||||
|
||||
+123
-37
@@ -51,13 +51,29 @@ static void AndroidLogHandler(const LogMessageEnvelope &env, const char *message
|
||||
}
|
||||
#endif
|
||||
|
||||
Model::Model(const char *model_path) {
|
||||
Model::Model(const char *model_path) : model_path_str_(model_path) {
|
||||
|
||||
#ifdef __ANDROID__
|
||||
SetLogHandler(AndroidLogHandler);
|
||||
#endif
|
||||
|
||||
const char *usage = "Read the docs";
|
||||
struct stat buffer;
|
||||
string am_path = model_path_str_ + "/am/final.mdl";
|
||||
if (stat(am_path.c_str(), &buffer) == 0) {
|
||||
ConfigureV2();
|
||||
} else {
|
||||
ConfigureV1();
|
||||
}
|
||||
|
||||
ReadDataFiles();
|
||||
|
||||
ref_cnt_ = 1;
|
||||
}
|
||||
|
||||
// Old model layout without model configuration file
|
||||
|
||||
void Model::ConfigureV1()
|
||||
{
|
||||
const char *extra_args[] = {
|
||||
"--min-active=200",
|
||||
"--max-active=3000",
|
||||
@@ -72,54 +88,95 @@ Model::Model(const char *model_path) {
|
||||
"--endpoint.rule3.min-trailing-silence=1.0",
|
||||
"--endpoint.rule4.min-trailing-silence=2.0",
|
||||
};
|
||||
std::string model_path_str(model_path);
|
||||
|
||||
kaldi::ParseOptions po(usage);
|
||||
kaldi::ParseOptions po("");
|
||||
nnet3_decoding_config_.Register(&po);
|
||||
endpoint_config_.Register(&po);
|
||||
decodable_opts_.Register(&po);
|
||||
|
||||
std::vector<const char*> args;
|
||||
vector<const char*> args;
|
||||
args.push_back("vosk");
|
||||
args.insert(args.end(), extra_args, extra_args + sizeof(extra_args) / sizeof(extra_args[0]));
|
||||
po.Read(args.size(), args.data());
|
||||
|
||||
feature_info_.feature_type = "mfcc";
|
||||
ReadConfigFromFile(model_path_str + "/mfcc.conf", &feature_info_.mfcc_opts);
|
||||
ReadConfigFromFile(model_path_str_ + "/mfcc.conf", &feature_info_.mfcc_opts);
|
||||
feature_info_.mfcc_opts.frame_opts.allow_downsample = true; // It is safe to downsample
|
||||
|
||||
feature_info_.silence_weighting_config.silence_weight = 1e-3;
|
||||
feature_info_.silence_weighting_config.silence_phones_str = "1:2:3:4:5:6:7:8:9:10";
|
||||
feature_info_.silence_weighting_config.silence_phones_str = endpoint_config_.silence_phones;
|
||||
|
||||
OnlineIvectorExtractionConfig ivector_extraction_opts;
|
||||
ivector_extraction_opts.splice_config_rxfilename = model_path_str + "/ivector/splice.conf";
|
||||
ivector_extraction_opts.cmvn_config_rxfilename = model_path_str + "/ivector/online_cmvn.conf";
|
||||
ivector_extraction_opts.lda_mat_rxfilename = model_path_str + "/ivector/final.mat";
|
||||
ivector_extraction_opts.global_cmvn_stats_rxfilename = model_path_str + "/ivector/global_cmvn.stats";
|
||||
ivector_extraction_opts.diag_ubm_rxfilename = model_path_str + "/ivector/final.dubm";
|
||||
ivector_extraction_opts.ivector_extractor_rxfilename = model_path_str + "/ivector/final.ie";
|
||||
ivector_extraction_opts.num_gselect = 5;
|
||||
ivector_extraction_opts.min_post = 0.025;
|
||||
ivector_extraction_opts.posterior_scale = 0.1;
|
||||
ivector_extraction_opts.max_remembered_frames = 1000;
|
||||
ivector_extraction_opts.max_count = 100;
|
||||
ivector_extraction_opts.ivector_period = 200;
|
||||
ivector_extraction_opts.splice_config_rxfilename = model_path_str_ + "/ivector/splice.conf";
|
||||
ivector_extraction_opts.cmvn_config_rxfilename = model_path_str_ + "/ivector/online_cmvn.conf";
|
||||
ivector_extraction_opts.lda_mat_rxfilename = model_path_str_ + "/ivector/final.mat";
|
||||
ivector_extraction_opts.global_cmvn_stats_rxfilename = model_path_str_ + "/ivector/global_cmvn.stats";
|
||||
ivector_extraction_opts.diag_ubm_rxfilename = model_path_str_ + "/ivector/final.dubm";
|
||||
ivector_extraction_opts.ivector_extractor_rxfilename = model_path_str_ + "/ivector/final.ie";
|
||||
feature_info_.use_ivectors = true;
|
||||
feature_info_.ivector_extractor_info.Init(ivector_extraction_opts);
|
||||
|
||||
std::string nnet3_rxfilename = model_path_str + "/final.mdl";
|
||||
std::string hclg_fst_rxfilename = model_path_str + "/HCLG.fst";
|
||||
std::string hcl_fst_rxfilename = model_path_str + "/HCLr.fst";
|
||||
std::string g_fst_rxfilename = model_path_str + "/Gr.fst";
|
||||
std::string disambig_rxfilename = model_path_str + "/disambig_tid.int";
|
||||
std::string word_syms_rxfilename = model_path_str + "/words.txt";
|
||||
std::string winfo_rxfilename = model_path_str + "/word_boundary.int";
|
||||
nnet3_rxfilename_ = model_path_str_ + "/final.mdl";
|
||||
hclg_fst_rxfilename_ = model_path_str_ + "/HCLG.fst";
|
||||
hcl_fst_rxfilename_ = model_path_str_ + "/HCLr.fst";
|
||||
g_fst_rxfilename_ = model_path_str_ + "/Gr.fst";
|
||||
disambig_rxfilename_ = model_path_str_ + "/disambig_tid.int";
|
||||
word_syms_rxfilename_ = model_path_str_ + "/words.txt";
|
||||
winfo_rxfilename_ = model_path_str_ + "/word_boundary.int";
|
||||
carpa_rxfilename_ = model_path_str_ + "/rescore/G.carpa";
|
||||
std_fst_rxfilename_ = model_path_str_ + "/rescore/G.fst";
|
||||
}
|
||||
|
||||
void Model::ConfigureV2()
|
||||
{
|
||||
kaldi::ParseOptions po("something");
|
||||
nnet3_decoding_config_.Register(&po);
|
||||
endpoint_config_.Register(&po);
|
||||
decodable_opts_.Register(&po);
|
||||
po.ReadConfigFile(model_path_str_ + "/conf/model.conf");
|
||||
|
||||
KALDI_LOG << "Decoding params beam=" << nnet3_decoding_config_.beam <<
|
||||
" max-active=" << nnet3_decoding_config_.max_active <<
|
||||
" lattice-beam=" << nnet3_decoding_config_.lattice_beam;
|
||||
KALDI_LOG << "Silence phones " << endpoint_config_.silence_phones;
|
||||
|
||||
feature_info_.feature_type = "mfcc";
|
||||
ReadConfigFromFile(model_path_str_ + "/conf/mfcc.conf", &feature_info_.mfcc_opts);
|
||||
feature_info_.mfcc_opts.frame_opts.allow_downsample = true; // It is safe to downsample
|
||||
|
||||
feature_info_.silence_weighting_config.silence_weight = 1e-3;
|
||||
feature_info_.silence_weighting_config.silence_phones_str = endpoint_config_.silence_phones;
|
||||
|
||||
OnlineIvectorExtractionConfig ivector_extraction_opts;
|
||||
ivector_extraction_opts.splice_config_rxfilename = model_path_str_ + "/ivector/splice.conf";
|
||||
ivector_extraction_opts.cmvn_config_rxfilename = model_path_str_ + "/ivector/online_cmvn.conf";
|
||||
ivector_extraction_opts.lda_mat_rxfilename = model_path_str_ + "/ivector/final.mat";
|
||||
ivector_extraction_opts.global_cmvn_stats_rxfilename = model_path_str_ + "/ivector/global_cmvn.stats";
|
||||
ivector_extraction_opts.diag_ubm_rxfilename = model_path_str_ + "/ivector/final.dubm";
|
||||
ivector_extraction_opts.ivector_extractor_rxfilename = model_path_str_ + "/ivector/final.ie";
|
||||
feature_info_.use_ivectors = true;
|
||||
feature_info_.ivector_extractor_info.Init(ivector_extraction_opts);
|
||||
|
||||
nnet3_rxfilename_ = model_path_str_ + "/am/final.mdl";
|
||||
hclg_fst_rxfilename_ = model_path_str_ + "/graph/HCLG.fst";
|
||||
hcl_fst_rxfilename_ = model_path_str_ + "/graph/HCLr.fst";
|
||||
g_fst_rxfilename_ = model_path_str_ + "/graph/Gr.fst";
|
||||
disambig_rxfilename_ = model_path_str_ + "/graph/disambig_tid.int";
|
||||
word_syms_rxfilename_ = model_path_str_ + "/graph/words.txt";
|
||||
winfo_rxfilename_ = model_path_str_ + "/graph/phones/word_boundary.int";
|
||||
carpa_rxfilename_ = model_path_str_ + "/rescore/G.carpa";
|
||||
std_fst_rxfilename_ = model_path_str_ + "/rescore/G.fst";
|
||||
}
|
||||
|
||||
void Model::ReadDataFiles()
|
||||
{
|
||||
struct stat buffer;
|
||||
|
||||
trans_model_ = new kaldi::TransitionModel();
|
||||
nnet_ = new kaldi::nnet3::AmNnetSimple();
|
||||
{
|
||||
bool binary;
|
||||
kaldi::Input ki(nnet3_rxfilename, &binary);
|
||||
kaldi::Input ki(nnet3_rxfilename_, &binary);
|
||||
trans_model_->Read(ki.Stream(), binary);
|
||||
nnet_->Read(ki.Stream(), binary);
|
||||
SetBatchnormTestMode(true, &(nnet_->GetNnet()));
|
||||
@@ -129,16 +186,17 @@ Model::Model(const char *model_path) {
|
||||
decodable_info_ = new nnet3::DecodableNnetSimpleLoopedInfo(decodable_opts_,
|
||||
nnet_);
|
||||
|
||||
struct stat buffer;
|
||||
if (stat(hclg_fst_rxfilename.c_str(), &buffer) == 0) {
|
||||
hclg_fst_ = fst::ReadFstKaldiGeneric(hclg_fst_rxfilename);
|
||||
if (stat(hclg_fst_rxfilename_.c_str(), &buffer) == 0) {
|
||||
KALDI_LOG << "Loading HCLG from " << hclg_fst_rxfilename_;
|
||||
hclg_fst_ = fst::ReadFstKaldiGeneric(hclg_fst_rxfilename_);
|
||||
hcl_fst_ = NULL;
|
||||
g_fst_ = NULL;
|
||||
} else {
|
||||
KALDI_LOG << "Loading HCL and G from " << hcl_fst_rxfilename_ << " " << g_fst_rxfilename_;
|
||||
hclg_fst_ = NULL;
|
||||
hcl_fst_ = fst::StdFst::Read(hcl_fst_rxfilename);
|
||||
g_fst_ = fst::StdFst::Read(g_fst_rxfilename);
|
||||
ReadIntegerVectorSimple(disambig_rxfilename, &disambig_);
|
||||
hcl_fst_ = fst::StdFst::Read(hcl_fst_rxfilename_);
|
||||
g_fst_ = fst::StdFst::Read(g_fst_rxfilename_);
|
||||
ReadIntegerVectorSimple(disambig_rxfilename_, &disambig_);
|
||||
}
|
||||
|
||||
word_syms_ = NULL;
|
||||
@@ -148,18 +206,46 @@ Model::Model(const char *model_path) {
|
||||
word_syms_ = g_fst_->OutputSymbols();
|
||||
}
|
||||
if (!word_syms_) {
|
||||
if (!(word_syms_ = fst::SymbolTable::ReadText(word_syms_rxfilename)))
|
||||
KALDI_LOG << "Loading words from " << word_syms_rxfilename_;
|
||||
if (!(word_syms_ = fst::SymbolTable::ReadText(word_syms_rxfilename_)))
|
||||
KALDI_ERR << "Could not read symbol table from file "
|
||||
<< word_syms_rxfilename;
|
||||
<< word_syms_rxfilename_;
|
||||
}
|
||||
KALDI_ASSERT(word_syms_);
|
||||
|
||||
if (stat(winfo_rxfilename.c_str(), &buffer) == 0) {
|
||||
if (stat(winfo_rxfilename_.c_str(), &buffer) == 0) {
|
||||
KALDI_LOG << "Loading winfo " << winfo_rxfilename_;
|
||||
kaldi::WordBoundaryInfoNewOpts opts;
|
||||
winfo_ = new kaldi::WordBoundaryInfo(opts, winfo_rxfilename);
|
||||
winfo_ = new kaldi::WordBoundaryInfo(opts, winfo_rxfilename_);
|
||||
} else {
|
||||
winfo_ = NULL;
|
||||
}
|
||||
|
||||
if (stat(carpa_rxfilename_.c_str(), &buffer) == 0) {
|
||||
KALDI_LOG << "Loading CARPA model from " << carpa_rxfilename_;
|
||||
std_lm_fst_ = fst::ReadFstKaldi(std_fst_rxfilename_);
|
||||
fst::Project(std_lm_fst_, fst::PROJECT_OUTPUT);
|
||||
if (std_lm_fst_->Properties(fst::kILabelSorted, true) == 0) {
|
||||
fst::ILabelCompare<fst::StdArc> ilabel_comp;
|
||||
fst::ArcSort(std_lm_fst_, ilabel_comp);
|
||||
}
|
||||
ReadKaldiObject(carpa_rxfilename_, &const_arpa_);
|
||||
} else {
|
||||
std_lm_fst_ = NULL;
|
||||
}
|
||||
}
|
||||
|
||||
void Model::Ref()
|
||||
{
|
||||
ref_cnt_++;
|
||||
}
|
||||
|
||||
void Model::Unref()
|
||||
{
|
||||
ref_cnt_--;
|
||||
if (ref_cnt_ == 0) {
|
||||
delete this;
|
||||
}
|
||||
}
|
||||
|
||||
Model::~Model() {
|
||||
|
||||
+25
-2
@@ -32,6 +32,7 @@
|
||||
#include "rnnlm/rnnlm-utils.h"
|
||||
|
||||
using namespace kaldi;
|
||||
using namespace std;
|
||||
|
||||
class KaldiRecognizer;
|
||||
|
||||
@@ -39,11 +40,28 @@ class Model {
|
||||
|
||||
public:
|
||||
Model(const char *model_path);
|
||||
~Model();
|
||||
void Ref();
|
||||
void Unref();
|
||||
|
||||
protected:
|
||||
~Model();
|
||||
void ConfigureV1();
|
||||
void ConfigureV2();
|
||||
void ReadDataFiles();
|
||||
|
||||
friend class KaldiRecognizer;
|
||||
|
||||
string model_path_str_;
|
||||
string nnet3_rxfilename_;
|
||||
string hclg_fst_rxfilename_;
|
||||
string hcl_fst_rxfilename_;
|
||||
string g_fst_rxfilename_;
|
||||
string disambig_rxfilename_;
|
||||
string word_syms_rxfilename_;
|
||||
string winfo_rxfilename_;
|
||||
string carpa_rxfilename_;
|
||||
string std_fst_rxfilename_;
|
||||
|
||||
kaldi::OnlineEndpointConfig endpoint_config_;
|
||||
kaldi::LatticeFasterDecoderConfig nnet3_decoding_config_;
|
||||
kaldi::nnet3::NnetSimpleLoopedComputationOptions decodable_opts_;
|
||||
@@ -54,11 +72,16 @@ protected:
|
||||
kaldi::nnet3::AmNnetSimple *nnet_;
|
||||
const fst::SymbolTable *word_syms_;
|
||||
kaldi::WordBoundaryInfo *winfo_;
|
||||
std::vector<int32> disambig_;
|
||||
vector<int32> disambig_;
|
||||
|
||||
fst::Fst<fst::StdArc> *hclg_fst_;
|
||||
fst::Fst<fst::StdArc> *hcl_fst_;
|
||||
fst::Fst<fst::StdArc> *g_fst_;
|
||||
|
||||
fst::VectorFst<fst::StdArc> *std_lm_fst_;
|
||||
kaldi::ConstArpaLm const_arpa_;
|
||||
|
||||
int ref_cnt_;
|
||||
};
|
||||
|
||||
#endif /* MODEL_H_ */
|
||||
|
||||
@@ -24,4 +24,19 @@ SpkModel::SpkModel(const char *speaker_path) {
|
||||
SetBatchnormTestMode(true, &speaker_nnet);
|
||||
SetDropoutTestMode(true, &speaker_nnet);
|
||||
CollapseModel(nnet3::CollapseModelConfig(), &speaker_nnet);
|
||||
|
||||
ref_cnt_ = 1;
|
||||
}
|
||||
|
||||
void SpkModel::Ref()
|
||||
{
|
||||
ref_cnt_++;
|
||||
}
|
||||
|
||||
void SpkModel::Unref()
|
||||
{
|
||||
ref_cnt_--;
|
||||
if (ref_cnt_ == 0) {
|
||||
delete this;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,12 +27,17 @@ class SpkModel {
|
||||
|
||||
public:
|
||||
SpkModel(const char *spk_path);
|
||||
void Ref();
|
||||
void Unref();
|
||||
|
||||
protected:
|
||||
friend class KaldiRecognizer;
|
||||
~SpkModel() {};
|
||||
|
||||
kaldi::nnet3::Nnet speaker_nnet;
|
||||
MfccOptions spkvector_mfcc_opts;
|
||||
|
||||
int ref_cnt_;
|
||||
};
|
||||
|
||||
#endif /* SPK_MODEL_H_ */
|
||||
|
||||
+1
-3
@@ -1,4 +1,4 @@
|
||||
%module(package="vosk") vosk
|
||||
%module(package="vosk", "threads"=1) vosk
|
||||
|
||||
%include <typemaps.i>
|
||||
|
||||
@@ -10,8 +10,6 @@
|
||||
%include <arrays_csharp.i>
|
||||
#endif
|
||||
|
||||
|
||||
|
||||
#if SWIGPYTHON
|
||||
%pybuffer_binary(const char *data, int len);
|
||||
#endif
|
||||
|
||||
+5
-5
@@ -28,7 +28,7 @@ VoskModel *vosk_model_new(const char *model_path)
|
||||
|
||||
void vosk_model_free(VoskModel *model)
|
||||
{
|
||||
delete (Model *)model;
|
||||
((Model *)model)->Unref();
|
||||
}
|
||||
|
||||
VoskSpkModel *vosk_spk_model_new(const char *model_path)
|
||||
@@ -38,22 +38,22 @@ VoskSpkModel *vosk_spk_model_new(const char *model_path)
|
||||
|
||||
void vosk_spk_model_free(VoskSpkModel *model)
|
||||
{
|
||||
delete (SpkModel *)model;
|
||||
((SpkModel *)model)->Unref();
|
||||
}
|
||||
|
||||
VoskRecognizer *vosk_recognizer_new(VoskModel *model, float sample_rate)
|
||||
{
|
||||
return (VoskRecognizer *)new KaldiRecognizer(*(Model *)model, sample_rate);
|
||||
return (VoskRecognizer *)new KaldiRecognizer((Model *)model, sample_rate);
|
||||
}
|
||||
|
||||
VoskRecognizer *vosk_recognizer_new_spk(VoskModel *model, VoskSpkModel *spk_model, float sample_rate)
|
||||
{
|
||||
return (VoskRecognizer *)new KaldiRecognizer(*(Model *)model, (SpkModel *)spk_model, sample_rate);
|
||||
return (VoskRecognizer *)new KaldiRecognizer((Model *)model, (SpkModel *)spk_model, sample_rate);
|
||||
}
|
||||
|
||||
VoskRecognizer *vosk_recognizer_new_grm(VoskModel *model, float sample_rate, const char *grammar)
|
||||
{
|
||||
return (VoskRecognizer *)new KaldiRecognizer(*(Model *)model, sample_rate, grammar);
|
||||
return (VoskRecognizer *)new KaldiRecognizer((Model *)model, sample_rate, grammar);
|
||||
}
|
||||
|
||||
int vosk_recognizer_accept_waveform(VoskRecognizer *recognizer, const char *data, int length)
|
||||
|
||||
@@ -35,7 +35,7 @@ RUN cd /opt \
|
||||
&& sed -i "s:-mfloat-abi=hard -mfpu=neon:${ARM_HARDWARE_OPTS}:g" makefiles/linux_openblas_arm.mk \
|
||||
&& sed -i "s: -O1 : -O3 :g" makefiles/linux_openblas_arm.mk \
|
||||
&& ./configure --mathlib=OPENBLAS --shared --use-cuda=no \
|
||||
&& make -j 10 online2 \
|
||||
&& make -j 10 online2 lm \
|
||||
&& find /opt/kaldi -name "*.o" -exec rm {} \;
|
||||
|
||||
RUN cd /opt \
|
||||
|
||||
@@ -27,7 +27,7 @@ RUN cd /opt \
|
||||
&& make -j 10 openfst \
|
||||
&& cd ../src \
|
||||
&& ./configure --mathlib=OPENBLAS --shared --use-cuda=no \
|
||||
&& make -j 10 online2 \
|
||||
&& make -j 10 online2 lm \
|
||||
&& find /opt/kaldi -name "*.o" -exec rm {} \;
|
||||
|
||||
RUN cd /opt \
|
||||
|
||||
@@ -3,11 +3,9 @@
|
||||
set -e
|
||||
set -x
|
||||
|
||||
skip() {
|
||||
docker build --build-arg="DOCKCROSS_IMAGE=linux-armv7" --build-arg="OPENBLAS_ARCH=ARMV7" --file Dockerfile.dockcross --tag alphacep/kaldi-dockcross-armv7:latest .
|
||||
docker build --build-arg="DOCKCROSS_IMAGE=linux-armv6" --build-arg="OPENBLAS_ARCH=ARMV6" --build-arg="ARM_HARDWARE_OPTS=" --file Dockerfile.dockcross --tag alphacep/kaldi-dockcross-armv6:latest .
|
||||
docker build --build-arg="DOCKCROSS_IMAGE=linux-arm64" --build-arg="OPENBLAS_ARCH=ARMV8" --file Dockerfile.dockcross --tag alphacep/kaldi-dockcross-arm64:latest .
|
||||
}
|
||||
|
||||
docker run --rm -v /home/shmyrev/travis/vosk-api/:/io alphacep/kaldi-dockcross-armv6 /io/travis/build-wheels-dockcross.sh
|
||||
docker run --rm -v /home/shmyrev/travis/vosk-api/:/io alphacep/kaldi-dockcross-armv7 /io/travis/build-wheels-dockcross.sh
|
||||
|
||||
Reference in New Issue
Block a user