docs: preserve upstream English README
Create PR to main with cherry-pick from release / cherry-pick (push) Failing after 0s
Build, validate, and release Neural Modules / pre-flight (push) Has been cancelled
Build, validate, and release Neural Modules / release (push) Has been cancelled
Build, validate, and release Neural Modules / release-summary (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Create PR to main with cherry-pick from release / cherry-pick (push) Failing after 0s
Build, validate, and release Neural Modules / pre-flight (push) Has been cancelled
Build, validate, and release Neural Modules / release (push) Has been cancelled
Build, validate, and release Neural Modules / release-summary (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
This commit is contained in:
+128
@@ -0,0 +1,128 @@
|
||||
[](http://www.repostatus.org/#active)
|
||||
[](https://docs.nvidia.com/nemo/speech/nightly/)
|
||||
[](https://github.com/nvidia/nemo/actions/workflows/codeql.yml)
|
||||
[](https://github.com/NVIDIA/NeMo/blob/master/LICENSE)
|
||||
[](https://badge.fury.io/py/nemo-toolkit)
|
||||
[](https://badge.fury.io/py/nemo-toolkit)
|
||||
[](https://pepy.tech/project/nemo-toolkit)
|
||||
[](https://github.com/psf/black)
|
||||
|
||||
# **NVIDIA NeMo Speech**
|
||||
Checkout our [HuggingFace🤗 collection](https://huggingface.co/collections/nvidia/nemotron-speech) for the latest open
|
||||
weight checkpoints and demos!
|
||||
|
||||
## Updates
|
||||
|
||||
> The first release of NeMo Speech after NeMo repository split is scheduled for June 2026, as the repo undergoes transformation.
|
||||
> For the latest stable released version, please use [the 26.02 NGC container](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo?version=26.02).
|
||||
|
||||
- 2026-06: [Nemotron-3.5-ASR-Streaming-0.6B](https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b) has been released with 40 languages supported, controllable latency 80ms-1s, and 240-2400 1xH100 concurrent streams. Built on cache-aware Fastconformer architecture.
|
||||
- 2026-04: [Parakeet-unified-en-0.6b](https://huggingface.co/nvidia/parakeet-unified-en-0.6b) has been released with high-quality offline and streaming (with a minimum latency of 160ms) inference in one model for English language with punctuation and capitalization support.
|
||||
- 2026-03: [Nemotron 3 VoiceChat](https://build.nvidia.com/nvidia/nemotron-voicechat/modelcard) is now released in Early Access. Built on the Nemotron Nano v2 LLM backbone with Nemotron speech and TTS decoder, VoiceChat delivers full-duplex, natural, interruptible conversations with low latency. Try out [the demo](https://build.nvidia.com/nvidia/nemotron-voicechat) and apply for [early access](https://developer.nvidia.com/nemotron-voicechat-early-access).
|
||||
- 2026-03: [Nemotron-Speech-Streaming v2603](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b) has been
|
||||
updated. It has been trained on a larger and more diverse corpus, resulting in lower WER across all latency modes.
|
||||
Try out [the demo](https://huggingface.co/spaces/nvidia/nemotron-speech-streaming-en-0.6b) and check out
|
||||
[the NIM](https://build.nvidia.com/nvidia/nemotron-asr-streaming).
|
||||
- 2026-03: [MagpieTTS v2602](https://huggingface.co/nvidia/magpie_tts_multilingual_357m) has been released with support
|
||||
for 9 languages(En, Es, De, Fr, Vi, It, Zh, Hi, Ja). Try out
|
||||
[the demo](https://huggingface.co/nvidia/magpie_tts_multilingual_357m) and check out
|
||||
[the NIM](https://build.nvidia.com/nvidia/magpie-tts-multilingual).
|
||||
- 2026-01: Nemotron-Speech-Streaming was released: One checkpoint that enables users to pick their optimal point
|
||||
on the latency-accuracy Pareto curve!
|
||||
- 2026-01: MagpieTTS was released.
|
||||
- 2026: This repo has pivoted to focus on audio, speech, and multimodal LLM. For the last NeMo release with support for more
|
||||
modalities, see [v2.7.0](https://github.com/NVIDIA-NeMo/NeMo/releases/tag/v2.7.0)
|
||||
- 2025-08: [Parakeet V3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) and
|
||||
[Canary V2](https://huggingface.co/nvidia/canary-1b-v2) have been released with speech recognition and translation
|
||||
support for 25 European languages.
|
||||
- 2025-06: [Canary-Qwen-2.5B](https://huggingface.co/nvidia/canary-qwen-2.5b) has been released with record-setting
|
||||
5.63% WER on English Open ASR Leaderboard.
|
||||
|
||||
## Introduction
|
||||
|
||||
NVIDIA NeMo Speech is built for researchers and PyTorch developers working on Speech models including Automatic Speech
|
||||
Recognition (ASR), Text to Speech (TTS), and Speech LLMs. It is designed to help you efficiently create, customize, and
|
||||
deploy new AI models by leveraging existing code and pre-trained model checkpoints.
|
||||
|
||||
For technical documentation, please see the
|
||||
[NeMo Framework User Guide](https://docs.nvidia.com/nemo/speech/nightly/).
|
||||
|
||||
## Requirements
|
||||
|
||||
NeMo Speech works with the **Python, PyTorch, and CUDA versions of your choosing**:
|
||||
|
||||
- Python 3.12 or above
|
||||
- PyTorch 2.7 or above (CPU, CUDA, etc. — your choice)
|
||||
- NVIDIA GPU + CUDA (required for training; recommended for inference)
|
||||
|
||||
If you already have a Python/PyTorch/CUDA stack that satisfies those minimums, NeMo Speech installs on top of it **without replacing it**, so your existing PyTorch build is kept (see the install options below). The versions pinned in `uv.lock` and shipped in the official container — Python 3.13, PyTorch 2.12, CUDA 12.6/13.2 — are simply the combination we actively test and support. They make setup turnkey and reproducible, but they are **not** a hard requirement.
|
||||
|
||||
As of [Pytorch 2.6](https://docs.pytorch.org/docs/stable/notes/serialization.html#torch-load-with-weights-only-true),
|
||||
`torch.load` defaults to using `weights_only=True`. Some model checkpoints may require using `weights_only=False`.
|
||||
In this case, you can set the env var `TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1` before running code that uses `torch.load`.
|
||||
However, this should only be done with trusted files. Loading files from untrusted sources with more than weights only
|
||||
can have the risk of arbitrary code execution.
|
||||
|
||||
## Developer Documentation
|
||||
|
||||
| Version | Status | Description |
|
||||
| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| Latest | [](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/) | [Documentation of the latest (i.e. main) branch.](https://docs.nvidia.com/nemo/speech/nightly/) |
|
||||
| Stable | [](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/) | Documentation of the stable (i.e. most recent release) - To be added |
|
||||
|
||||
## Install NeMo Speech
|
||||
|
||||
The recommended way to install NeMo Speech is from source with [uv](https://docs.astral.sh/uv/), which reproduces our actively-tested stack from the committed `uv.lock`. If you need different Python/PyTorch/CUDA versions, NeMo also installs over your existing environment via pip — see the [pip fallback](#from-pypi-with-pip-fallback--bring-your-own-versions) below.
|
||||
|
||||
### From source with uv (recommended)
|
||||
|
||||
```bash
|
||||
git clone https://github.com/NVIDIA-NeMo/NeMo.git
|
||||
cd NeMo
|
||||
uv sync --extra all --extra cu13 # CUDA 13.x (recommended) — use --extra cu12 for CUDA 12.x
|
||||
```
|
||||
|
||||
This installs our supported stack (Python 3.13, PyTorch 2.12, CUDA 13.2) into `.venv/` with NeMo editable. Add `--group test` for the test suite or `--group docs` to build the docs; run tools via `uv run <cmd>` or activate with `source .venv/bin/activate`. On Linux, `cu12` and `cu13` are mutually exclusive — pass exactly one (`cu13` is the default). For the **exact** container baseline, add `--locked --python 3.13` (the path the Dockerfile and CI use).
|
||||
|
||||
> **SpeechLM2 / Automodel:** the Automodel backend runs **without** any compiled dependencies. It can *optionally* benefit from dedicated accelerated backends (Transformer Engine, FlashAttention, Mamba, grouped-GEMM/MoE, DeepEP) for better performance — these source-built kernels come from the `compiled` (Hopper/Blackwell) or `compiled-a100` (A100) extras, built by `docker/Dockerfile` (`GPU_TARGET=h100plus` / `a100`). See the [installation guide](https://docs.nvidia.com/nemo/speech/nightly/) for the full list and build details.
|
||||
|
||||
### Docker (turnkey, our supported stack)
|
||||
|
||||
> **NGC container:** _Coming soon — the pull command for the prebuilt NeMo Speech container image will be published here._
|
||||
|
||||
To build the container from source (CUDA 13 / H100+ by default):
|
||||
|
||||
```bash
|
||||
git clone https://github.com/NVIDIA-NeMo/NeMo.git
|
||||
cd NeMo
|
||||
docker buildx build -f docker/Dockerfile -t nemo-speech . # CUDA 13 / H100+ (default)
|
||||
docker run --rm -it --gpus all -v "$PWD:/workspace" nemo-speech bash
|
||||
```
|
||||
|
||||
For A100, set `GPU_TARGET=a100` — A100 works with **both CUDA 12 and CUDA 13** (CUDA 13, the default base image, is recommended; the CUDA 12 base is a convenience). See the header of [`docker/Dockerfile`](docker/Dockerfile) for all build arguments (`BASE_IMAGE`, `GPU_TARGET`).
|
||||
|
||||
### From PyPI with pip (fallback — bring your own versions)
|
||||
|
||||
Prefer your own Python/PyTorch/CUDA? Install your PyTorch first (any version ≥ 2.7 for your CPU/CUDA/etc. target — see the [PyTorch install matrix](https://pytorch.org/get-started/locally/)), then add NeMo and it **keeps your build**. `uv pip` (uv's fast, pip-compatible installer) works like `pip`:
|
||||
|
||||
```bash
|
||||
uv pip install 'nemo-toolkit[asr,tts]' # or plain: pip install 'nemo-toolkit[asr,tts]'
|
||||
```
|
||||
|
||||
> ⚠️ Do **not** use `uv sync --locked` for a bring-your-own stack — it applies `uv.lock` and replaces your Python/PyTorch/CUDA with the supported baseline. Use `uv pip`/`pip` here; reserve `uv sync --locked` for reproducing our stack.
|
||||
|
||||
To instead pull *our* pinned PyTorch build, add the CUDA extra and the matching wheel index (pip/uv pip do not read uv's project index config, so `--extra-index-url` is required):
|
||||
|
||||
```bash
|
||||
pip install 'nemo-toolkit[asr,tts,cu13]' --extra-index-url https://download.pytorch.org/whl/cu132 # CUDA 13.x
|
||||
pip install 'nemo-toolkit[asr,tts,cu12]' --extra-index-url https://download.pytorch.org/whl/cu126 # CUDA 12.x
|
||||
```
|
||||
|
||||
## Contribute to NeMo
|
||||
|
||||
We welcome community contributions! Please refer to
|
||||
[CONTRIBUTING.md](https://github.com/NVIDIA-NeMo/NeMo/blob/main/CONTRIBUTING.md) for the process.
|
||||
|
||||
## Licenses
|
||||
|
||||
NeMo is licensed under the [Apache License 2.0](https://github.com/NVIDIA/NeMo?tab=Apache-2.0-1-ov-file).
|
||||
Reference in New Issue
Block a user