chore: import upstream snapshot with attribution

docs/source/starthere/choosing_a_model.rst

@@ -0,0 +1,111 @@
+.. _choosing-a-model:
+
+Choosing a Model
+================
+
+NeMo offers many pretrained speech models. This guide helps you pick the right one for your use case.
+
+ASR: Which Model Should I Use?
+------------------------------
+
+.. list-table::
+   :widths: 30 25 45
+   :header-rows: 1
+
+   * - I want to...
+     - Recommended Model
+     - Why
+   * - Get the best accuracy on English
+     - `Canary-Qwen 2.5B <https://huggingface.co/nvidia/canary-qwen-2.5b>`_
+     - State-of-the-art English ASR. For very fast offline alternatives with almost SOTA accuracy, use `Parakeet-TDT V2 <https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2>`_ or `Parakeet-TDT V3 <https://huggingface.co/nvidia/parakeet-tdt-1.1b>`_.
+   * - Transcribe multiple languages
+     - `Canary-1B V2 <https://huggingface.co/nvidia/canary-1b-v2>`_
+     - Supports 25 EU languages + translation between them. AED decoder.
+   * - Transcribe European languages (ASR only, auto language detection)
+     - `Parakeet-TDT 0.6B V3 <https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3>`_
+     - 25 European languages in one model; automatic language detection; punctuation, capitalization, and word/segment timestamps; long-form and streaming options. No speech-to-text translation—use Canary-1B V2 if you need translation.
+   * - Stream audio in real-time
+     - `Nemotron-3.5-ASR-Streaming <https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b>`_
+     - Low-latency streaming ASR with 40 languages, controllable latency (80ms–1s), and configurable chunk sizes. Cache-aware FastConformer.
+   * - Minimize model size
+     - `Canary-180M Flash <https://huggingface.co/nvidia/canary-180m-flash>`_
+     - Smallest multilingual model. Good for edge deployment.
+   * - Use CTC decoding (simpler pipeline)
+     - `Parakeet-CTC-1.1B <https://huggingface.co/nvidia/parakeet-ctc-1.1b>`_
+     - Non-autoregressive. Fast. Good with external language models.
+   * - Integrate with an external LM
+     - Any Parakeet model + NGPU-LM
+     - GPU-accelerated n-gram LM fusion for CTC, RNNT, and TDT models.
+   * - Transcribe multi-speaker meetings
+     - `Multitalker Parakeet Streaming <https://huggingface.co/nvidia/multitalker-parakeet-streaming-0.6b-v1>`_
+     - Handles overlapping speech in real-time with speaker-adapted decoding.
+
+TTS: Which Model Should I Use?
+------------------------------
+
+.. list-table::
+   :widths: 30 25 45
+   :header-rows: 1
+
+   * - I want to...
+     - Recommended Model
+     - Why
+   * - Generate high-quality multilingual speech
+     - `MagpieTTS <https://huggingface.co/nvidia/magpie_tts_multilingual_357m>`_
+     - End-to-end LLM-based TTS. Supports voice cloning and multiple languages.
+   * - Fast, controllable English synthesis
+     - `FastPitch <https://huggingface.co/nvidia/tts_en_fastpitch>`_ + `HiFi-GAN <https://huggingface.co/nvidia/tts_hifigan>`_
+     - Cascaded pipeline with pitch/duration control. Well-tested.
+   * - Generate discrete audio tokens
+     - Audio Codec
+     - Neural audio codec for tokenizing audio. Used by MagpieTTS internally.
+
+Speaker Tasks: Which Model Should I Use?
+-----------------------------------------
+
+.. list-table::
+   :widths: 30 25 45
+   :header-rows: 1
+
+   * - I want to...
+     - Recommended Model
+     - Why
+   * - Determine who spoke when
+     - `Streaming Sortformer <https://huggingface.co/nvidia/diar_streaming_sortformer_4spk-v2.1>`_, `Offline Sortformer <https://huggingface.co/nvidia/diar_sortformer_4spk-v1>`_
+     - End-to-end diarization for up to 4 speakers. Use streaming for real-time; use offline for batch.
+   * - Verify/identify a speaker
+     - `TitaNet <https://huggingface.co/nvidia/speakerverification_en_titanet_large>`_
+     - Extracts speaker embeddings for verification and identification.
+   * - Detect voice activity
+     - `MarbleNet <https://huggingface.co/nvidia/Frame_VAD_Multilingual_MarbleNet_v2.0>`_
+     - Frame-level VAD. Multilingual. Works as a preprocessing step.
+
+Speech Language Models: Which Model Should I Use?
+-------------------------------------------------
+
+.. list-table::
+   :widths: 30 25 45
+   :header-rows: 1
+
+   * - I want to...
+     - Recommended Model
+     - Why
+   * - Ask questions about audio content
+     - `Canary-Qwen 2.5B <https://huggingface.co/nvidia/canary-qwen-2.5b>`_ (SALM)
+     - LLM augmented with speech understanding. Can transcribe, translate, and answer questions about audio.
+   * - Build a speech-to-speech system
+     - DuplexS2SModel
+     - Full-duplex model that both understands and generates speech.
+
+
+Where to Find Models
+--------------------
+
+All pretrained NeMo models are available on:
+
+- `HuggingFace Hub (nvidia) <https://huggingface.co/nvidia>`_ — search for "nemo" or specific model names
+- `NGC Model Catalog <https://catalog.ngc.nvidia.com/models?query=nemo&orderBy=weightPopularDESC>`_ — NVIDIA's model registry
+- :doc:`Featured Community Checkpoints </asr/featured_community_checkpoints>` — fine-tunes from external users
+
+See :doc:`../checkpoints/intro` for instructions on loading pretrained models.
+

docs/source/starthere/install.rst

@@ -0,0 +1,210 @@
+.. _installation:
+
+Installation
+============
+
+This page covers how to install NVIDIA NeMo for speech AI tasks (ASR, TTS, speaker tasks, audio processing, and speech language models).
+
+Prerequisites
+-------------
+
+NeMo Speech works with the **Python, PyTorch, and CUDA versions of your choosing**:
+
+#. **Python** 3.12 or above
+#. **PyTorch** 2.7 or above, for your chosen target (CPU, CUDA, etc.)
+#. **NVIDIA GPU + CUDA** (required for training; CPU-only inference is possible but slow)
+#. **uv** for the fastest source/PyPI workflow (``pip`` also works in a prepared environment)
+
+.. admonition:: Bring your own Python / PyTorch / CUDA
+   :class: important
+
+   The recommended install path is uv (below), which gives you our actively-tested stack. But NeMo Speech can also install *on top of* an existing environment: if you already have a Python, PyTorch, and CUDA stack that satisfies the minimums above, your pre-installed PyTorch is **kept, not replaced** (see :ref:`the pip fallback <install-from-pypi>`).
+
+   The versions pinned in ``uv.lock`` and shipped in the official container — **Python 3.13, PyTorch 2.12, CUDA 12.6/13.2** — are simply the combination we actively test and support. They make setup turnkey and reproducible, but they are **not** a hard requirement.
+
+.. note::
+
+   As of `PyTorch 2.6 <https://docs.pytorch.org/docs/stable/notes/serialization.html#torch-load-with-weights-only-true>`_, ``torch.load`` defaults to ``weights_only=True``. Some checkpoints require ``weights_only=False``; in that case set ``TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1`` before loading, and only with trusted files (loading untrusted files with full pickle support risks arbitrary code execution).
+
+.. _install-from-source:
+
+Install from Source with uv (recommended)
+------------------------------------------
+
+The recommended way to install NeMo Speech is from source with `uv <https://docs.astral.sh/uv/>`_, which reproduces our actively-tested stack from the committed ``uv.lock``:
+
+.. code-block:: bash
+
+   git clone https://github.com/NVIDIA-NeMo/NeMo.git
+   cd NeMo
+
+   # CUDA 13.x (recommended). Use --extra cu12 for CUDA 12.x. uv resolves the
+   # matching PyTorch CUDA wheel automatically from the pinned indexes.
+   uv sync --extra all --extra cu13
+
+   # Optional: add the test suite tooling, or the docs build dependencies
+   # uv sync --extra all --extra cu13 --group test
+   # uv sync --group docs
+
+``uv sync`` creates a virtual environment in ``.venv/`` with NeMo installed in editable mode, matching our supported stack (Python 3.13, PyTorch 2.12, CUDA 13.2 by default). Run commands with ``uv run <cmd>`` or activate the environment with ``source .venv/bin/activate``. For the **exact** container baseline, add ``--locked --python 3.13`` (i.e. ``uv sync --locked --python 3.13 --extra all --extra cu13``) — this is the path the Dockerfile and CI use.
+
+On Linux, pass exactly one of ``--extra cu13`` (recommended) or ``--extra cu12`` — they are mutually exclusive. If you omit both, uv installs the generic PyPI PyTorch wheel instead of NVIDIA's CUDA-matched build.
+
+Available collection extras (combine with one CUDA extra above):
+
+.. list-table::
+   :widths: 18 82
+   :header-rows: 1
+
+   * - Extra
+     - What it includes
+   * - ``asr``
+     - Automatic Speech Recognition models, data loaders, and utilities
+   * - ``tts``
+     - Text-to-Speech models, vocoders, and audio codecs
+   * - ``audio``
+     - Audio processing models (enhancement, separation)
+   * - ``speechlm2``
+     - Speech language models (includes NeMo Automodel)
+   * - ``all``
+     - All of the collections above
+   * - ``cu12`` / ``cu13``
+     - Our pinned CUDA 12.x / 13.x PyTorch build **plus** the matching CUDA Python deps (``cuda-python``, ``numba-cuda``). Linux; pick at most one.
+
+.. note::
+
+   ``test`` and ``docs`` are dependency *groups* (PEP 735), not extras. Install them with ``--group`` (e.g. ``uv sync --group test``) — the bracket form ``.[test]`` does not work.
+
+.. _install-compiled-extras:
+
+Optional compiled dependencies for SpeechLM2 / Automodel (``compiled`` / ``compiled-a100``)
+-------------------------------------------------------------------------------------------
+
+The Automodel backend used for SpeechLM2 **does not require any compiled dependencies — it runs without them.** The ``compiled`` and ``compiled-a100`` extras are an *optional* performance add-on: when their source-built GPU kernels are installed, Automodel can route to dedicated accelerated backends (FP8 Transformer kernels via Transformer Engine, FlashAttention, Mamba/state-space layers, and Mixture-of-Experts ops). They contain:
+
+.. list-table::
+   :widths: 30 70
+   :header-rows: 1
+
+   * - Package
+     - Purpose
+   * - ``transformer-engine``
+     - NVIDIA Transformer Engine — FP8 and accelerated Transformer kernels
+   * - ``flash-attn``
+     - FlashAttention attention kernels
+   * - ``mamba-ssm`` + ``causal-conv1d``
+     - Mamba / state-space-model kernels (hybrid Mamba architectures)
+   * - ``nv-grouped-gemm``
+     - Grouped GEMM kernels for Mixture-of-Experts (MoE) layers
+   * - ``deep_ep`` (DeepEP)
+     - Expert-parallel communication kernels for MoE (``compiled`` only — see below)
+   * - ``onnx-ir`` + ``onnxscript``
+     - Pinned ONNX export tooling
+
+Choose the variant that matches your GPU (the two are mutually exclusive):
+
+* ``compiled`` — Hopper/Blackwell and newer (SM90/SM100/SM120, e.g. H100/H200/B200). Includes DeepEP.
+* ``compiled-a100`` — Ampere A100 (SM80). Omits DeepEP, which requires a separately-built, patched version on A100; our Dockerfile auto-builds and installs it when the CUDA 12 base image is selected.
+
+.. warning::
+
+   These packages **build from source** and need a full CUDA build environment — build tools, matching ``TORCH_CUDA_ARCH_LIST`` / ``NVTE_CUDA_ARCHS`` flags, ``--no-build-isolation``, and (for ``compiled``) extra manual build steps that the Dockerfile performs (e.g. flash-attn-4 and DeepEP patches). The supported, reproducible way to get them is the container build, which sets all of this up for you:
+
+   .. code-block:: bash
+
+      # Hopper/Blackwell (default GPU_TARGET=h100plus → compiled)
+      docker buildx build -f docker/Dockerfile -t nemo-speech .
+
+      # Ampere A100 (GPU_TARGET=a100 → compiled-a100)
+      docker buildx build -f docker/Dockerfile \
+        --build-arg BASE_IMAGE=nvcr.io/nvidia/cuda-dl-base:25.06-cuda12.9-devel-ubuntu24.04 \
+        --build-arg GPU_TARGET=a100 -t nemo-speech .
+
+   A bare ``uv sync --extra all --extra cu13 --extra compiled`` outside this environment will likely fail to compile.
+
+Using Docker (turnkey, our supported stack)
+--------------------------------------------
+
+.. note::
+
+   **NGC container:** *Coming soon — the pull command for the prebuilt NeMo Speech container image will be published here.*
+
+To build the container from source, use the provided ``docker/Dockerfile`` (CUDA 13 / H100+ by default):
+
+.. code-block:: bash
+
+   git clone https://github.com/NVIDIA-NeMo/NeMo.git
+   cd NeMo
+   docker buildx build -f docker/Dockerfile -t nemo-speech .          # CUDA 13 / H100+ (default)
+   docker run --rm -it --gpus all -v "$PWD:/workspace" nemo-speech bash
+
+For A100, set ``GPU_TARGET=a100``. A100 works with **both CUDA 12 and CUDA 13** — CUDA 13 (the default base image) is recommended; the CUDA 12 base is offered only as a convenience:
+
+.. code-block:: bash
+
+   # A100 on CUDA 13 (recommended) — uses the default CUDA 13 base image
+   docker buildx build -f docker/Dockerfile --build-arg GPU_TARGET=a100 -t nemo-speech:a100 .
+
+   # A100 on CUDA 12 (convenience)
+   docker buildx build -f docker/Dockerfile \
+     --build-arg BASE_IMAGE=nvcr.io/nvidia/cuda-dl-base:25.06-cuda12.9-devel-ubuntu24.04 \
+     --build-arg GPU_TARGET=a100 -t nemo-speech:a100-cu12 .
+
+See the header of ``docker/Dockerfile`` for all build arguments (``BASE_IMAGE``, ``GPU_TARGET``).
+
+.. _install-from-pypi:
+
+Install from PyPI with pip (fallback — bring your own versions)
+---------------------------------------------------------------
+
+Prefer your own Python/PyTorch/CUDA? Install your preferred PyTorch first (any version ≥ 2.7 for your CPU/CUDA/etc. target — see `PyTorch's install matrix <https://pytorch.org/get-started/locally/>`_), then add NeMo. Your pre-installed PyTorch is kept, not replaced. ``uv pip`` (uv's fast, pip-compatible installer) works just like ``pip``:
+
+.. code-block:: bash
+
+   uv venv --python 3.12          # any Python >= 3.12 your PyTorch supports — or use your own env
+   source .venv/bin/activate
+
+   # 1) Your choice of PyTorch (example: CUDA 12.6 build). Skip if you already have one.
+   uv pip install torch --index-url https://download.pytorch.org/whl/cu126
+
+   # 2) NeMo — your PyTorch above is kept (plain `pip install` works identically)
+   uv pip install 'nemo-toolkit[asr,tts]'        # also: [asr,tts,audio], [speechlm2], etc.
+
+.. warning::
+
+   Do **not** use ``uv sync --locked`` for a bring-your-own stack — it intentionally applies ``uv.lock`` and replaces your Python/PyTorch/CUDA with the supported container baseline. Use ``uv pip`` (or ``pip``) here; reserve ``uv sync --locked`` for reproducing the supported stack (above).
+
+To instead have the installer pull *our* pinned PyTorch build, add the matching CUDA extra **and** the PyTorch wheel index (``pip`` / ``uv pip`` do not read uv's project index config, so ``--extra-index-url`` is required):
+
+.. code-block:: bash
+
+   pip install 'nemo-toolkit[asr,tts,cu13]' --extra-index-url https://download.pytorch.org/whl/cu132   # CUDA 13.x
+   pip install 'nemo-toolkit[asr,tts,cu12]' --extra-index-url https://download.pytorch.org/whl/cu126   # CUDA 12.x
+
+.. tip::
+
+   Prefer a conda environment? Create and activate one (``conda create -n nemo python=3.12 -y && conda activate nemo``), then run the same ``uv`` or ``pip`` commands above inside it. NeMo Speech does not require a separate conda CUDA toolkit.
+
+Verify Installation
+-------------------
+
+After installing, verify that the chosen collection imports:
+
+.. code-block:: bash
+
+   python -c "import nemo.collections.asr as nemo_asr; print('NeMo ASR installed')"
+
+If you installed with ``uv sync`` and have not activated ``.venv``, run the check through ``uv run python``. To also exercise a model download:
+
+.. code-block:: python
+
+   import nemo.collections.asr as nemo_asr
+   model = nemo_asr.models.ASRModel.from_pretrained("nvidia/parakeet-tdt-0.6b-v2")
+   print(f"Loaded: {model.__class__.__name__}")
+
+What's Next?
+------------
+
+- :doc:`ten_minutes` — A quick tour of NeMo's speech capabilities
+- :doc:`key_concepts` — Understand the fundamentals of speech AI
+- :doc:`choosing_a_model` — Find the right model for your use case

docs/source/starthere/key_concepts.rst

@@ -0,0 +1,192 @@
+.. _key-concepts:
+
+Key Concepts in Speech AI
+=========================
+
+This page introduces the fundamental concepts you'll encounter when working with speech models in NeMo. No prior NeMo experience is required — we start from the basics of audio and work up to how NeMo structures its models.
+
+Audio Conventions in NeMo
+-------------------------
+
+**Sampling rate** — ASR models often use **16 kHz**; TTS and audio processing models may use higher rates (e.g. 22.05 kHz, 44.1 kHz). Check each model's or preprocessor's config for the expected sample rate.
+
+**Channels** — Most models use mono input, but some support **multi-channel** audio (e.g. for spatial or multi-mic setups). See the model and preprocessor documentation for your use case.
+
+**Preprocessing** — NeMo models typically include a **preprocessor** that converts waveform input into features (e.g. mel-spectrogram). For most setups, you should provide audio that already matches the model's expected **sample rate** and **channel layout** (often mono); automatic resampling or stereo→mono is not guaranteed and depends on the collection, dataset, and preprocessor config. Check the model and preprocessor documentation for your use case.
+
+**Mel-spectrogram** — For models that use it, the preprocessor turns raw waveform into mel-spectrogram features; this is handled inside the model, not as a separate offline step.
+
+
+Speech AI Tasks
+---------------
+
+NeMo supports several speech AI tasks, each solving a different problem:
+
+.. list-table::
+   :widths: 20 40 40
+   :header-rows: 1
+
+   * - Task
+     - What it does
+     - Example use case
+   * - **ASR** (Automatic Speech Recognition)
+     - Converts spoken audio to text
+     - Transcribing meetings, voice interfaces
+   * - **TTS** (Text-to-Speech)
+     - Generates natural speech from text
+     - Audiobooks, voice interfaces
+   * - **Speaker Diarization**
+     - Determines "who spoke when"
+     - Multi-speaker segmentation and transcription
+   * - **Speaker Recognition**
+     - Identifies or verifies a speaker's identity
+     - Voice authentication, speaker search
+   * - **Speech Enhancement**
+     - Improves audio quality (removes noise)
+     - Preprocessing noisy recordings
+   * - **SpeechLM**
+     - Augments LLMs with audio understanding
+     - Audio-aware agents, speech translation, reasoning about audio
+
+
+Encoder Architectures
+---------------------
+
+The *encoder* converts audio features into a sequence of high-level representations:
+
+**Transformer**
+   The standard encoder from `Vaswani et al. (2017) <https://arxiv.org/abs/1706.03762>`_ — stacked self-attention and feed-forward layers with no convolutions. Used in NeMo as an encoder or decoder in encoder-decoder models (e.g. Canary).
+
+**Conformer**
+   The original architecture from `Gulati et al. (2020) <https://arxiv.org/abs/2005.08100>`_ that combines self-attention with convolutions for both global and local patterns.
+
+**FastConformer**
+   A faster variant of Conformer (`Rekesh et al. (2023) <https://arxiv.org/abs/2305.05084>`_) with 8× subsampling and optimized attention. NeMo's default choice for ASR; recommended for new projects.
+
+
+How NeMo Models Work
+---------------------
+
+Every NeMo model wraps these components into a single, cohesive unit:
+
+.. raw:: html
+
+   <div style="margin: 24px 0; overflow-x: auto;">
+   <svg viewBox="0 0 820 130" xmlns="http://www.w3.org/2000/svg" style="max-width:820px; width:100%; height:auto; font-family:'NVIDIA Sans',sans-serif;">
+     <defs>
+       <marker id="arrow" viewBox="0 0 10 10" refX="10" refY="5" markerWidth="8" markerHeight="8" orient="auto"><path d="M0,0 L10,5 L0,10 z" fill="#76b900"/></marker>
+     </defs>
+     <!-- Preprocessor -->
+     <rect x="0" y="20" width="140" height="70" rx="8" fill="#76b900" opacity="0.15" stroke="#76b900" stroke-width="2"/>
+     <text x="70" y="48" text-anchor="middle" font-weight="700" font-size="13" fill="#333">Preprocessor</text>
+     <text x="70" y="66" text-anchor="middle" font-size="10" fill="#555">Audio &#8594; Mel-spectrogram</text>
+     <!-- Arrow 1 -->
+     <line x1="140" y1="55" x2="170" y2="55" stroke="#76b900" stroke-width="2" marker-end="url(#arrow)"/>
+     <!-- Encoder -->
+     <rect x="170" y="20" width="140" height="70" rx="8" fill="#76b900" opacity="0.15" stroke="#76b900" stroke-width="2"/>
+     <text x="240" y="48" text-anchor="middle" font-weight="700" font-size="13" fill="#333">Encoder</text>
+     <text x="240" y="66" text-anchor="middle" font-size="10" fill="#555">Features &#8594; Hidden repr.</text>
+     <!-- Arrow 2 -->
+     <line x1="310" y1="55" x2="340" y2="55" stroke="#76b900" stroke-width="2" marker-end="url(#arrow)"/>
+     <!-- Decoder -->
+     <rect x="340" y="20" width="140" height="70" rx="8" fill="#76b900" opacity="0.15" stroke="#76b900" stroke-width="2"/>
+     <text x="410" y="48" text-anchor="middle" font-weight="700" font-size="13" fill="#333">Decoder</text>
+     <text x="410" y="66" text-anchor="middle" font-size="10" fill="#555">Hidden repr. &#8594; Output</text>
+     <!-- Arrow 3 -->
+     <line x1="480" y1="55" x2="510" y2="55" stroke="#76b900" stroke-width="2" marker-end="url(#arrow)"/>
+     <!-- Loss -->
+     <rect x="510" y="20" width="140" height="70" rx="8" fill="#76b900" opacity="0.15" stroke="#76b900" stroke-width="2"/>
+     <text x="580" y="48" text-anchor="middle" font-weight="700" font-size="13" fill="#333">Loss Function</text>
+     <text x="580" y="66" text-anchor="middle" font-size="10" fill="#555">Measures quality</text>
+     <!-- Arrow 4 -->
+     <line x1="650" y1="55" x2="680" y2="55" stroke="#76b900" stroke-width="2" marker-end="url(#arrow)"/>
+     <!-- Optimizer -->
+     <rect x="680" y="20" width="140" height="70" rx="8" fill="#76b900" opacity="0.15" stroke="#76b900" stroke-width="2"/>
+     <text x="750" y="48" text-anchor="middle" font-weight="700" font-size="13" fill="#333">Optimizer</text>
+     <text x="750" y="66" text-anchor="middle" font-size="10" fill="#555">Updates weights</text>
+   </svg>
+   </div>
+
+
+Overview of NeMo Speech
+========================
+
+NeMo models are PyTorch modules that also integrate with `PyTorch Lightning <https://lightning.ai/>`__ for training and `Hydra <https://hydra.cc/>`__ + `OmegaConf <https://omegaconf.readthedocs.io/>`__ for configuration.
+
+Configuration with YAML
+------------------------
+
+NeMo experiments are configured with YAML files. A typical config has three main sections:
+
+.. code-block:: yaml
+
+   model:
+     # Model architecture, data, loss, optimizer
+     encoder:
+       _target_: nemo.collections.asr.modules.ConformerEncoder
+       feat_in: 80
+       n_layers: 17
+       ...
+     train_ds:
+       manifest_filepath: /path/to/train_manifest.json
+       batch_size: 32
+     optim:
+       name: adamw
+       lr: 0.001
+
+   trainer:
+     # PyTorch Lightning trainer settings
+     devices: 4
+     accelerator: gpu
+     max_steps: 100000
+     precision: bf16-mixed
+
+   exp_manager:
+     # Experiment logging and checkpointing
+     exp_dir: /path/to/experiments
+     name: my_asr_experiment
+
+You can override any value from the command line:
+
+.. code-block:: bash
+
+   python train_script.py \
+       model.optim.lr=0.0005 \
+       model.train_ds.manifest_filepath=/data/train.json \
+       trainer.devices=8
+
+
+Manifest Files
+--------------
+
+NeMo uses **manifest files** (JSONL format) to describe datasets. Each line is one training example:
+
+.. code-block:: json
+
+   {"audio_filepath": "/data/audio/001.wav", "text": "hello world", "duration": 2.5}
+   {"audio_filepath": "/data/audio/002.wav", "text": "how are you", "duration": 1.8}
+
+Key fields:
+
+- ``audio_filepath`` — path to the audio file
+- ``text`` — the transcript (for ASR) or input text (for TTS)
+- ``duration`` — audio duration in seconds
+
+See :doc:`../asr/datasets` for details on preparing datasets.
+
+
+Model Checkpoints
+-----------------
+
+NeMo models are saved as ``.nemo`` files — tar archives containing model weights, configuration, and tokenizer files. You can load models in two ways:
+
+.. code-block:: python
+
+   # From a pretrained checkpoint (downloads from HuggingFace/NGC)
+   model = nemo_asr.models.ASRModel.from_pretrained("nvidia/parakeet-tdt-0.6b-v2")
+
+   # From a local .nemo file
+   model = nemo_asr.models.ASRModel.restore_from("path/to/model.nemo")
+
+See :doc:`../checkpoints/intro` for more details on checkpoint formats.
+

docs/source/starthere/ten_minutes.rst

@@ -0,0 +1,120 @@
+.. _ten-minutes:
+
+NeMo Speech Inference in 5 Minutes
+===================================
+
+This guide gives you a quick, hands-on tour of NeMo's core speech capabilities. By the end, you'll have transcribed audio, synthesized speech, identified speakers, and used a speech language model — all in about 50 lines of code.
+
+.. note::
+
+   Make sure you have :doc:`installed NeMo <install>` before starting.
+
+
+1. Transcribe Speech (ASR)
+--------------------------
+
+Automatic Speech Recognition converts audio to text. NeMo's Parakeet model sits at the top of the `HuggingFace OpenASR Leaderboard <https://huggingface.co/spaces/hf-audio/open_asr_leaderboard>`_.
+
+**Basic transcription** — 3 lines of code:
+
+.. code-block:: python
+
+   import nemo.collections.asr as nemo_asr
+
+   asr_model = nemo_asr.models.ASRModel.from_pretrained("nvidia/parakeet-tdt-0.6b-v2")
+   transcript = asr_model.transcribe(["audio.wav"])[0].text
+   print(transcript)
+
+**With timestamps** — know *when* each word was spoken:
+
+.. code-block:: python
+
+   hypotheses = asr_model.transcribe(["audio.wav"], timestamps=True)
+   for stamp in hypotheses[0].timestamp['word']:
+       print(f"{stamp['start']}s - {stamp['end']}s : {stamp['word']}")
+
+**From the command line**:
+
+.. code-block:: bash
+
+   python examples/asr/transcribe_speech.py \
+       pretrained_name="nvidia/parakeet-tdt-0.6b-v2" \
+       audio_dir=./my_audio_files/
+
+
+2. Synthesize Speech (TTS)
+--------------------------
+
+Text-to-Speech generates natural audio from text. NeMo's **Magpie TTS** is a multilingual, codec-based model that supports multiple speakers and languages:
+
+.. code-block:: python
+
+   from nemo.collections.tts.models import MagpieTTSModel
+   import soundfile as sf
+
+   # Load model (multilingual 357M, from Hugging Face)
+   model = MagpieTTSModel.from_pretrained("nvidia/magpie_tts_multilingual_357m")
+   model.eval()
+
+   # Generate speech
+   audio, audio_len = model.do_tts(
+       transcript="Hello! Welcome to NeMo speech AI.",
+       language="en",
+   )
+
+   # Save to file
+   sf.write("output.wav", audio[0].cpu().numpy(), 22050)
+   print("Speech saved to output.wav")
+
+
+3. Identify Speakers (Diarization)
+----------------------------------
+
+Speaker diarization answers "who spoke when?" in multi-speaker audio.
+
+.. code-block:: python
+
+   from nemo.collections.asr.models import SortformerEncLabelModel
+
+   diar_model = SortformerEncLabelModel.from_pretrained("nvidia/diar_streaming_sortformer_4spk-v2")
+   diar_model.eval()
+
+   segments = diar_model.diarize(audio=["meeting.wav"], batch_size=1)
+   for seg in segments[0]:
+       print(seg)  # (begin_seconds, end_seconds, speaker_index)
+
+
+4. Speech Language Models (SpeechLM2)
+-------------------------------------
+
+SpeechLM2 augments large language models with speech understanding. Canary-Qwen combines an ASR encoder with a Qwen LLM:
+
+.. code-block:: python
+
+   from nemo.collections.speechlm2.models import SALM
+
+   model = SALM.from_pretrained('nvidia/canary-qwen-2.5b')
+
+   answer_ids = model.generate(
+       prompts=[[{
+           "role": "user",
+           "content": f"Transcribe the following: {model.audio_locator_tag}",
+           "audio": ["speech.wav"],
+       }]],
+       max_new_tokens=128,
+   )
+   print(model.tokenizer.ids_to_text(answer_ids[0].cpu()))
+
+
+What's Next?
+------------
+
+Now that you've seen the basics, dive deeper:
+
+- :doc:`key_concepts` — Understand the speech AI fundamentals behind these models
+- :doc:`choosing_a_model` — Find the best model for your specific use case
+- :doc:`../asr/intro` — Full ASR documentation
+- :doc:`../tts/intro` — Full TTS documentation
+- :doc:`../asr/speaker_diarization/intro` — Speaker diarization and recognition
+- :doc:`../starthere/tutorials` — Tutorial notebooks
+

docs/source/starthere/tutorials.rst

@@ -0,0 +1,184 @@
+.. _tutorials:
+
+Tutorials
+=========
+
+The best way to get started with NeMo is to start with one of our tutorials. These tutorials cover various domains and provide both introductory and advanced topics. They are designed to help you understand and use the NeMo toolkit effectively.
+
+Running Tutorials on Colab
+--------------------------
+
+Most NeMo tutorials can be run on `Google's Colab <https://colab.research.google.com/notebooks/intro.ipynb>`_.
+
+To run a tutorial:
+
+1. Click the **Colab** link associated with the tutorial you are interested in from the table below.
+2. Once in Colab, connect to an instance with a GPU by clicking **Runtime** > **Change runtime type** and selecting **GPU** as the hardware accelerator.
+
+Tutorial Overview
+-----------------
+
+.. list-table:: **General Tutorials**
+   :widths: 15 25 60
+   :header-rows: 1
+
+   * - Domain
+     - Title
+     - GitHub URL
+   * - General
+     - Getting Started: NeMo Fundamentals
+     - `NeMo Fundamentals <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/00_NeMo_Primer.ipynb>`_
+   * - General
+     - Getting Started: Audio translator example
+     - `Audio translator example <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/AudioTranslationSample.ipynb>`_
+   * - General
+     - Getting Started: Voice swap example
+     - `Voice swap example <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/VoiceSwapSample.ipynb>`_
+   * - General
+     - Getting Started: NeMo Models
+     - `NeMo Models <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/01_NeMo_Models.ipynb>`_
+   * - General
+     - Getting Started: NeMo Adapters
+     - `NeMo Adapters <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/02_NeMo_Adapters.ipynb>`_
+   * - General
+     - Getting Started: NeMo Models on Hugging Face Hub
+     - `NeMo Models on HF Hub <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/Publish_NeMo_Model_On_Hugging_Face_Hub.ipynb>`_
+
+.. list-table:: **Automatic Speech Recognition (ASR) Tutorials**
+   :widths: 15 30 55
+   :header-rows: 1
+
+   * - Domain
+     - Title
+     - GitHub URL
+   * - ASR
+     - ASR with NeMo
+     - `ASR with NeMo <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/ASR_with_NeMo.ipynb>`_
+   * - ASR
+     - ASR with Subword Tokenization
+     - `ASR with Subword Tokenization <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/ASR_with_Subword_Tokenization.ipynb>`_
+   * - ASR
+     - Offline ASR
+     - `Offline ASR <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/Offline_ASR.ipynb>`_
+   * - ASR
+     - Online ASR Microphone Cache Aware Streaming
+     - `Online ASR Microphone Cache Aware Streaming <https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Online_ASR_Microphone_Demo_Cache_Aware_Streaming.ipynb>`_
+   * - ASR
+     - Online ASR Microphone Buffered Streaming
+     - `Online ASR Microphone Buffered Streaming <https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Online_ASR_Microphone_Demo_Buffered_Streaming.ipynb>`_
+   * - ASR
+     - ASR CTC Language Fine-Tuning
+     - `ASR CTC Language Fine-Tuning <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/ASR_CTC_Language_Finetuning.ipynb>`_
+   * - ASR
+     - Intro to Transducers
+     - `Intro to Transducers <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/Intro_to_Transducers.ipynb>`_
+   * - ASR
+     - ASR with Transducers
+     - `ASR with Transducers <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/ASR_with_Transducers.ipynb>`_
+   * - ASR
+     - ASR with Adapters
+     - `ASR with Adapters <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/asr_adapters/ASR_with_Adapters.ipynb>`_
+   * - ASR
+     - Speech Commands
+     - `Speech Commands <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/Speech_Commands.ipynb>`_
+   * - ASR
+     - Online Offline Microphone Speech Commands
+     - `Online Offline Microphone Speech Commands <https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Online_Offline_Speech_Commands_Demo.ipynb>`_
+   * - ASR
+     - Voice Activity Detection
+     - `Voice Activity Detection <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/Voice_Activity_Detection.ipynb>`_
+   * - ASR
+     - Online Offline Microphone VAD
+     - `Online Offline Microphone VAD <https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Online_Offline_Microphone_VAD_Demo.ipynb>`_
+   * - ASR
+     - Speaker Recognition and Verification
+     - `Speaker Recognition and Verification <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/Speaker_Identification_Verification.ipynb>`_
+   * - ASR
+     - Speaker Diarization Inference
+     - `Speaker Diarization Inference <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/Speaker_Diarization_Inference.ipynb>`_
+   * - ASR
+     - ASR with Speaker Diarization
+     - `ASR with Speaker Diarization <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/ASR_with_SpeakerDiarization.ipynb>`_
+   * - ASR
+     - Online Noise Augmentation
+     - `Online Noise Augmentation <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/Online_Noise_Augmentation.ipynb>`_
+   * - ASR
+     - ASR for Telephony Speech
+     - `ASR for Telephony Speech <https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/ASR_for_telephony_speech.ipynb>`_
+   * - ASR
+     - Streaming inference
+     - `Streaming inference <https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Streaming_ASR.ipynb>`_
+   * - ASR
+     - Buffered Transducer inference
+     - `Buffered Transducer inference <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/Buffered_Transducer_Inference.ipynb>`_
+   * - ASR
+     - Buffered Transducer inference with LCS Merge
+     - `Buffered Transducer inference with LCS Merge <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/Buffered_Transducer_Inference_with_LCS_Merge.ipynb>`_
+   * - ASR
+     - Offline ASR with VAD for CTC models
+     - `Offline ASR with VAD for CTC models <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/Offline_ASR_with_VAD_for_CTC_models.ipynb>`_
+   * - ASR
+     - Self-supervised Pre-training for ASR
+     - `Self-supervised Pre-training for ASR <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/Self_Supervised_Pre_Training.ipynb>`_
+   * - ASR
+     - Multi-lingual ASR
+     - `Multi-lingual ASR <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/Multilang_ASR.ipynb>`_
+   * - ASR
+     - ASR Confidence Estimation
+     - `ASR Confidence Estimation <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/asr/ASR_Confidence_Estimation.ipynb>`_
+
+.. list-table:: **Text-to-Speech (TTS) Tutorials**
+   :widths: 15 35 50
+   :header-rows: 1
+
+   * - Domain
+     - Title
+     - GitHub URL
+   * - TTS
+     - Basic and Advanced: NeMo TTS Primer
+     - `NeMo TTS Primer <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tts/NeMo_TTS_Primer.ipynb>`_
+   * - TTS
+     - Basic and Advanced: TTS Speech/Text Aligner Inference
+     - `TTS Speech/Text Aligner Inference <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tts/Aligner_Inference_Examples.ipynb>`_
+   * - TTS
+     - Basic and Advanced: FastPitch and MixerTTS Model Training
+     - `FastPitch and MixerTTS Model Training <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tts/FastPitch_MixerTTS_Training.ipynb>`_
+   * - TTS
+     - Basic and Advanced: FastPitch Finetuning
+     - `FastPitch Finetuning <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tts/FastPitch_Finetuning.ipynb>`_
+   * - TTS
+     - Basic and Advanced: FastPitch and HiFiGAN Model Training for German
+     - `FastPitch and HiFiGAN Model Training for German <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tts/FastPitch_GermanTTS_Training.ipynb>`_
+   * - TTS
+     - Basic and Advanced: Tacotron2 Model Training
+     - `Tacotron2 Model Training <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tts/Tacotron2_Training.ipynb>`_
+   * - TTS
+     - Basic and Advanced: FastPitch Duration and Pitch Control
+     - `FastPitch Duration and Pitch Control <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tts/Inference_DurationPitchControl.ipynb>`_
+   * - TTS
+     - Basic and Advanced: FastPitch Speaker Interpolation
+     - `FastPitch Speaker Interpolation <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tts/FastPitch_Speaker_Interpolation.ipynb>`_
+   * - TTS
+     - Basic and Advanced: TTS Inference and Model Selection
+     - `TTS Inference and Model Selection <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tts/Inference_ModelSelect.ipynb>`_
+   * - TTS
+     - Basic and Advanced: TTS Pronunciation Customization
+     - `TTS Pronunciation Customization <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tts/Pronunciation_customization.ipynb>`_
+
+.. list-table:: **Tools and Utilities**
+   :widths: 15 25 60
+   :header-rows: 1
+
+   * - Domain
+     - Title
+     - GitHub URL
+   * - Utility Tools
+     - Utility Tools for Speech and Text: NeMo Forced Aligner
+     - `NeMo Forced Aligner <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tools/NeMo_Forced_Aligner_Tutorial.ipynb>`_
+   * - Utility Tools
+     - Utility Tools for Speech and Text: Speech Data Explorer
+     - `Speech Data Explorer <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tools/SDE_HowTo_v2.ipynb>`_
+   * - Utility Tools
+     - Utility Tools for Speech and Text: CTC Segmentation
+     - `CTC Segmentation <https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/tools/CTC_Segmentation_Tutorial.ipynb>`_
+