chore: import upstream snapshot with attribution
Create PR to main with cherry-pick from release / cherry-pick (push) Failing after 0s
CICD NeMo / pre-flight (push) Failing after 0s
CICD NeMo / configure (push) Has been skipped
Build, validate, and release Neural Modules / pre-flight (push) Failing after 1s
CICD NeMo / code-linting (push) Has been skipped
Build, validate, and release Neural Modules / release (push) Has been skipped
Build, validate, and release Neural Modules / release-summary (push) Has been cancelled
CICD NeMo / cicd-test-container-build (push) Has been cancelled
CICD NeMo / cicd-import-tests (push) Has been cancelled
CICD NeMo / L0_Setup_Test_Data_And_Models (push) Has been cancelled
CICD NeMo / cicd-main-unit-tests (push) Has been cancelled
CICD NeMo / cicd-main-speech (push) Has been cancelled
CICD NeMo / Nemo_CICD_Test (push) Has been cancelled
CICD NeMo / Coverage (e2e) (push) Has been cancelled
CICD NeMo / Coverage (unit-test) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
CICD NeMo / cicd-wait-in-queue (push) Has been cancelled
Create PR to main with cherry-pick from release / cherry-pick (push) Failing after 0s
CICD NeMo / pre-flight (push) Failing after 0s
CICD NeMo / configure (push) Has been skipped
Build, validate, and release Neural Modules / pre-flight (push) Failing after 1s
CICD NeMo / code-linting (push) Has been skipped
Build, validate, and release Neural Modules / release (push) Has been skipped
Build, validate, and release Neural Modules / release-summary (push) Has been cancelled
CICD NeMo / cicd-test-container-build (push) Has been cancelled
CICD NeMo / cicd-import-tests (push) Has been cancelled
CICD NeMo / L0_Setup_Test_Data_And_Models (push) Has been cancelled
CICD NeMo / cicd-main-unit-tests (push) Has been cancelled
CICD NeMo / cicd-main-speech (push) Has been cancelled
CICD NeMo / Nemo_CICD_Test (push) Has been cancelled
CICD NeMo / Coverage (e2e) (push) Has been cancelled
CICD NeMo / Coverage (unit-test) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
CICD NeMo / cicd-wait-in-queue (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,115 @@
|
||||
NeMo Speech Classification Configuration Files
|
||||
================================================
|
||||
|
||||
This page covers NeMo configuration file setup that is specific to models in the Speech Classification collection.
|
||||
For general information about how to set up and run experiments that is common to all NeMo models (e.g.
|
||||
experiment manager and PyTorch Lightning trainer parameters), see the :doc:`../../core/core` page.
|
||||
|
||||
The model section of NeMo Speech Classification configuration files will generally require information about the dataset(s) being
|
||||
used, the preprocessor for audio files, parameters for any augmentation being performed, as well as the
|
||||
model architecture specification.
|
||||
The sections on this page cover each of these in more detail.
|
||||
|
||||
Example configuration files for all of the NeMo ASR scripts can be found in the
|
||||
``<NeMo_git_root>/examples/asr/conf>``.
|
||||
|
||||
|
||||
Dataset Configuration
|
||||
---------------------
|
||||
|
||||
Training, validation, and test parameters are specified using the ``train_ds``, ``validation_ds``, and
|
||||
``test_ds`` sections of your configuration file, respectively.
|
||||
Depending on the task, you may have arguments specifying the sample rate of your audio files, labels, whether or not to shuffle the dataset, and so on.
|
||||
You may also decide to leave fields such as the ``manifest_filepath`` blank, to be specified via the command line
|
||||
at runtime.
|
||||
|
||||
Any initialization parameters that are accepted for the Dataset class used in your experiment
|
||||
can be set in the config file.
|
||||
See the :ref:`Datasets <asr-api-datasets>` section of the API for a list of Datasets and their respective parameters.
|
||||
|
||||
An example Speech Classification train and validation configuration could look like:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
model:
|
||||
sample_rate: 16000
|
||||
repeat: 2 # number of convolutional sub-blocks within a block, R in <MODEL>_[BxRxC]
|
||||
dropout: 0.0
|
||||
kernel_size_factor: 1.0
|
||||
labels: ['bed', 'bird', 'cat', 'dog', 'down', 'eight', 'five', 'four', 'go', 'happy', 'house', 'left', 'marvin',
|
||||
'nine', 'no', 'off', 'on', 'one', 'right', 'seven', 'sheila', 'six', 'stop', 'three', 'tree', 'two', 'up',
|
||||
'wow', 'yes', 'zero']
|
||||
|
||||
train_ds:
|
||||
manifest_filepath: ???
|
||||
sample_rate: ${model.sample_rate}
|
||||
labels: ${model.labels} # Uses the labels above
|
||||
batch_size: 128
|
||||
shuffle: True
|
||||
|
||||
validation_ds:
|
||||
manifest_filepath: ???
|
||||
sample_rate: ${model.sample_rate}
|
||||
labels: ${model.labels} # Uses the labels above
|
||||
batch_size: 128
|
||||
shuffle: False # No need to shuffle the validation data
|
||||
|
||||
If you would like to use tarred dataset, have a look at :ref:`Datasets Configuration <asr-configs-dataset-configuration>`.
|
||||
|
||||
|
||||
Preprocessor Configuration
|
||||
--------------------------
|
||||
Preprocessor helps to compute MFCC or mel spectrogram features that are given as inputs to model.
|
||||
For details on how to write this section, refer to :ref:`Preprocessor Configuration <asr-configs-preprocessor-configuration>`
|
||||
|
||||
Check config yaml files in ``<NeMo_git_root>/examples/asr/conf`` to find the processors been used by speech classification models.
|
||||
|
||||
|
||||
Augmentation Configurations
|
||||
---------------------------
|
||||
|
||||
There are a few on-the-fly spectrogram augmentation options for NeMo ASR, which can be specified by the
|
||||
configuration file using the ``augmentor`` and ``spec_augment`` section.
|
||||
For details on how to write this section, refer to the ASR :ref:`Augmentation Configuration <asr-configs-augmentation-configurations>` section.
|
||||
|
||||
Check config yaml files in ``<NeMo_git_root>/tutorials/asr/conf`` to find the processors been used by speech classification models.
|
||||
|
||||
|
||||
Model Architecture Configurations
|
||||
---------------------------------
|
||||
|
||||
Each configuration file should describe the model architecture being used for the experiment.
|
||||
Models in the NeMo ASR collection need a ``encoder`` section and a ``decoder`` section, with the ``_target_`` field
|
||||
specifying the module to use for each.
|
||||
|
||||
The following sections go into more detail about the specific configurations of each model architecture.
|
||||
|
||||
The :ref:`MatchboxNet <MarbleNet_model>` and :ref:`MarbleNet <MarbleNet_model>` models are very similar, and as
|
||||
such the components in their configs are very similar as well.
|
||||
|
||||
Decoder Configurations
|
||||
------------------------
|
||||
|
||||
After features have been computed from ConvASREncoder, we pass the features to decoder to compute embeddings and then to compute log_probs
|
||||
for training models.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
model:
|
||||
...
|
||||
decoder:
|
||||
_target_: nemo.collections.asr.modules.ConvASRDecoderClassification
|
||||
feat_in: *enc_final_filters
|
||||
return_logits: true # return logits if true, else return softmax output
|
||||
pooling_type: 'avg' # AdaptiveAvgPool1d 'avg' or AdaptiveMaxPool1d 'max'
|
||||
|
||||
|
||||
Fine-tuning Execution Flow Diagram
|
||||
----------------------------------
|
||||
|
||||
When preparing your own training or fine-tuning scripts, please follow the execution flow diagram order for correct inference.
|
||||
|
||||
Depending on the type of model, there may be extra steps that must be performed -
|
||||
|
||||
* Speech Classification models - `Examples directory for Classification Models <https://github.com/NVIDIA/NeMo/blob/stable/examples/asr/speech_classification/README.md>`_
|
||||
|
||||
Reference in New Issue
Block a user