wehub-resource-sync
ba4be087d5
CICD NeMo / cicd-main-unit-tests (push) Blocked by required conditions
CICD NeMo / cicd-main-speech (push) Blocked by required conditions
CICD NeMo / cicd-test-container-build (push) Blocked by required conditions
CICD NeMo / cicd-import-tests (push) Blocked by required conditions
CICD NeMo / L0_Setup_Test_Data_And_Models (push) Blocked by required conditions
CICD NeMo / Nemo_CICD_Test (push) Blocked by required conditions
CICD NeMo / Coverage (e2e) (push) Blocked by required conditions
CICD NeMo / Coverage (unit-test) (push) Blocked by required conditions
CodeQL / Analyze (python) (push) Waiting to run
Create PR to main with cherry-pick from release / cherry-pick (push) Failing after 0s
CICD NeMo / pre-flight (push) Failing after 0s
CICD NeMo / configure (push) Has been skipped
Build, validate, and release Neural Modules / pre-flight (push) Failing after 1s
CICD NeMo / code-linting (push) Has been skipped
CICD NeMo / cicd-wait-in-queue (push) Waiting to run
Build, validate, and release Neural Modules / release (push) Has been skipped
Build, validate, and release Neural Modules / release-summary (push) Has been cancelled
51 lines
2.5 KiB
Bash
51 lines
2.5 KiB
Bash
# Copyright (c) 2020-2025, NVIDIA CORPORATION.
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
|
|
# Run training with SALMAutomodel
|
|
torchrun --nproc-per-node 1 --no-python \
|
|
coverage run -a --data-file=/workspace/.coverage --source=/workspace/nemo \
|
|
examples/speechlm2/salm_train.py --config-name=salm_automodel \
|
|
model.pretrained_llm=/home/TestData/speechlm/pretrained_models/TinyLlama--TinyLlama_v1.1 \
|
|
model.pretrained_asr=/home/TestData/speechlm/pretrained_models/canary-1b-flash.nemo \
|
|
data.train_ds.input_cfg.0.cuts_path=/home/TestData/speechlm/lhotse/libri/librispeech_cuts_lower_train-clean-5.jsonl.gz \
|
|
data.validation_ds.datasets.val_set_0.input_cfg.0.cuts_path=/home/TestData/speechlm/lhotse/libri/librispeech_cuts_lower_dev-clean-2.jsonl.gz \
|
|
trainer.devices=1 \
|
|
trainer.max_steps=10
|
|
|
|
# Convert to HF format
|
|
coverage run -a --data-file=/workspace/.coverage --source=/workspace/nemo \
|
|
examples/speechlm2/to_hf.py \
|
|
class_path=nemo.collections.speechlm2.models.SALMAutomodel \
|
|
ckpt_path=salm_results/checkpoints/step\=10-last.ckpt \
|
|
ckpt_config=salm_results/exp_config.yaml \
|
|
output_dir=test_salm_automodel_hf_model
|
|
|
|
# Run generation (auto-detects SALMAutomodel from config.json)
|
|
coverage run -a --data-file=/workspace/.coverage --source=/workspace/nemo \
|
|
examples/speechlm2/salm_generate.py \
|
|
pretrained_name=test_salm_automodel_hf_model \
|
|
inputs=/home/TestData/speechlm/lhotse/libri/librispeech_cuts_lower_dev-clean-2-first10.jsonl.gz \
|
|
batch_size=4 \
|
|
output_manifest=generations_automodel.jsonl
|
|
head generations_automodel.jsonl
|
|
|
|
# Run generation + WER eval (auto-detects SALMAutomodel from config.json)
|
|
coverage run -a --data-file=/workspace/.coverage --source=/workspace/nemo \
|
|
examples/speechlm2/salm_eval.py \
|
|
pretrained_name=test_salm_automodel_hf_model \
|
|
inputs=/home/TestData/speechlm/lhotse/libri/librispeech_cuts_lower_dev-clean-2-first10.jsonl.gz \
|
|
batch_size=4 \
|
|
output_manifest=generations_automodel_wer.jsonl
|
|
head generations_automodel_wer.jsonl
|