<jupyter_start><jupyter_text>**MiniASR Tutorial: LibriSpeech Training**This is a tutorial for training an end-to-end automatic speech recognition model with the toolkit [MiniASR](https://github.com/vectominist/MiniASR).  You can run this notebook on [Google Colab](colab.research.google.com/), but to train an ASR model completely requires a Pro account since it needs several hours to converge.  **Download Code & Install Dependencies**Ref: [MiniASR](https://github.com/vectominist/MiniASR)<jupyter_code>! git clone https://github.com/paulhuangkm/MiniASR.git
% cd MiniASR
! pip3 install -e ./<jupyter_output>Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///home/jonathanhsu/MiniASR/example/data
[31mERROR: file:///home/jonathanhsu/MiniASR/example/data does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.[0m<jupyter_text>**Download Data**- training set: [Libri-light](https://github.com/facebookresearch/libri-light) fine-tuning set (10 hours, 0.6G)- development set: [LibriSpeech](https://www.openslr.org/12) `dev-clean` set- testing set: [LibriSpeech](https://www.openslr.org/12) `test-clean` set<jupyter_code>! mkdir -p data
%cd data
! wget https://dl.fbaipublicfiles.com/librilight/data/librispeech_finetuning.tgz
! tar zxf librispeech_finetuning.tgz
! rm librispeech_finetuning.tgz
! wget https://www.openslr.org/resources/12/dev-clean.tar.gz
! wget https://www.openslr.org/resources/12/test-clean.tar.gz
! tar zxf dev-clean.tar.gz
! tar zxf test-clean.tar.gz
! rm dev-clean.tar.gz
! rm test-clean.tar.gz
%cd ..<jupyter_output>--2022-02-28 16:06:38--  https://www.openslr.org/resources/12/dev-clean.tar.gz
Resolving www.openslr.org (www.openslr.org)... 172.16.58.3
Connecting to www.openslr.org (www.openslr.org)|172.16.58.3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 337926286 (322M) [application/x-gzip]
Saving to: ‘dev-clean.tar.gz’

dev-clean.tar.gz    100%[===================>] 322.27M   686KB/s    in 6m 31s  

2022-02-28 16:13:11 (844 KB/s) - ‘dev-clean.tar.gz’ saved [337926286/337926286]

--2022-02-28 16:13:11--  https://www.openslr.org/resources/12/test-clean.tar.gz
Resolving www.openslr.org (www.openslr.org)... 172.16.58.3
Connecting to www.openslr.org (www.openslr.org)|172.16.58.3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 346663984 (331M) [application/x-gzip]
Saving to: ‘test-clean.tar.gz’

test-clean.tar.gz   100%[===================>] 330.60M  1.90MB/s    in 5m 32s  

2022-02-28 16:18:44 (1020 KB/s) - ‘test-clean.tar.gz’ saved [3466[...]<jupyter_text>**Preprocess Data**Find all data in the corpus and extract vocabularies. We use characters as text tokens since the dataset is small.<jupyter_code># Train set
! miniasr-preprocess \
        -c LibriSpeech \
        -p data/librispeech_finetuning \
        -s 1h \
        -o data/libri_train_1h \
        --gen-vocab \
        --char-vocab-size 40

! miniasr-preprocess \
        -c LibriSpeech \
        -p data/librispeech_finetuning \
        -s 9h \
        -o data/libri_train_9h

# Development set
! miniasr-preprocess \
        -c LibriSpeech \
        -p data/LibriSpeech \
        -s dev-clean \
        -o data/libri_dev

# Test set
! miniasr-preprocess \
        -c LibriSpeech \
        -p data/LibriSpeech \
        -s test-clean \
        -o data/libri_test<jupyter_output>02-28 16:19 run_preprocess.py.main(72) Preprocessing LibriSpeech corpus.
02-28 16:19 run_preprocess.py.main(73) Subsets = ['1h']
02-28 16:19 run_preprocess.py.main(76) Results will be saved to data/libri_train_1h
02-28 16:19 run_preprocess.py.main(79) Reading data from data/librispeech_finetuning
02-28 16:19 run_preprocess.py.main(85) Found 286 audio files.
02-28 16:19 run_preprocess.py.main(89) Saving unsorted data dict to data/libri_train_1h/data_dict.json
02-28 16:19 run_preprocess.py.main(94) Sorting data by audio file length.
02-28 16:19 run_preprocess.py.main(103) Saving sorted data list to data/libri_train_1h/data_list_sorted.json
02-28 16:19 run_preprocess.py.main(109) Generating LM file.
02-28 16:19 run_preprocess.py.main(117) Generating vocabularies.
02-28 16:19 run_preprocess.py.main(120) Generating characters.
Found 28 chars.
Selected 28 vocabularies.
Saving char vocabularies to data/libri_train_1h/vocab_char.txt
02-28 16:19 run_preprocess.py.main(72) Preprocessing LibriSpe[...]<jupyter_text>**Training**- Modify `MiniASR/egs/librispeech/config/ctc_train_example.yaml` for changing training hyper-parameters.- The results will be saved to `MiniASR/model/ctc_libri-10h_char`.<jupyter_code>! mkdir -p model
! pwd
! minasr-asr --config ../egs/librispeech/config/con_ctc.yaml

# Resume training with this command:
# ! minasr-asr --ckpt model/con_libri-10h_char/epoch=4-step=429.ckpt<jupyter_output>02-28 16:28 run_asr.py.main(99) Training mode.
02-28 16:28 dataloader.py.create_dataloader(86) Creating text tokenizer of character level.
02-28 16:28 dataloader.py.create_dataloader(91) Generating datasets and dataloaders. (mode = train)
02-28 16:28 dataset.py.__init__(28) Loading data from ['data/libri_train_1h/data_list_sorted.json', 'data/libri_train_9h/data_list_sorted.json']
100%|████████████████████████████████████| 2763/2763 [00:00<00:00, 31793.95it/s]
02-28 16:28 dataset.py.__init__(49) 2763 audio files found (mode = train)
02-28 16:28 dataset.py.__init__(28) Loading data from ['data/libri_dev/data_list_sorted.json']
100%|████████████████████████████████████| 2703/2703 [00:00<00:00, 52287.83it/s]
02-28 16:28 dataset.py.__init__(49) 2703 audio files found (mode = dev)
02-28 16:28 asr_trainer.py.create_asr_trainer(24) Creating ASR model (type = con_asr).
Downloading: "https://github.com/s3prl/s3prl/archive/master.zip" to /home/jonathanhsu/.cache/torch/hub/master.zip
[s3prl.upstr[...]<jupyter_text>**Testing**- Specify your checkpoint with `--ckpt`.<jupyter_code>! minasr-asr \
    --config ../egs/librispeech/config/con_ctc_test.yaml \
    --test \
    --override "args.data.dev_paths=['data/libri_test/data_list_sorted.json']" \
    --ckpt model/con_libri-10h_char/epoch=174-step=15224.ckpt<jupyter_output>02-28 18:36 basic_setups.py.override(89) Override: args.data.dev_paths = ['data/libri_test/data_list_sorted.json']
02-28 18:36 run_asr.py.main(105) Testing mode.
[s3prl.upstream.experts] Warning: can not import s3prl.upstream.wav2vec2.expert: No module named 'fairseq'. Pass.
[s3prl.upstream.experts] Warning: can not import s3prl.upstream.hubert.expert: No module named 'fairseq'. Pass.
[s3prl.upstream.experts] Warning: can not import s3prl.upstream.distiller.expert: No module named 'fairseq'. Pass.
[s3prl.upstream.experts] Warning: can not import s3prl.upstream.wav2vec.expert: No module named 'fairseq'. Pass.
[s3prl.upstream.experts] Warning: can not import s3prl.upstream.roberta.expert: No module named 'fairseq'. Pass.
[s3prl.upstream.experts] Warning: can not import s3prl.upstream.vq_wav2vec.expert: No module named 'fairseq'. Pass.
[s3prl.upstream.experts] Warning: can not import s3prl.upstream.decoar2.expert: No module named 'fairseq'. Pass.
[s3prl.upstream.experts] Warning: can not [...]<jupyter_text>**Inference**<jupyter_code>from miniasr.utils import load_from_checkpoint, sequence_distance
from miniasr.data.audio import load_waveform

model, args, tokenizer = load_from_checkpoint(
    'model/ctc_libri-10h_char/epoch=44-step=3869.ckpt', 'cuda')
waves = [load_waveform('data/LibriSpeech/dev-clean/6345/93302/6345-93302-0025.flac').to('cuda')]
hyps = model.recognize(waves)
print(hyps[0])
ref = 'ARE YOU REALLY GOING TO THROW ME OVER FOR A THING LIKE THIS'
res_cer = sequence_distance(ref, hyps[0], mode='char')
res_wer = sequence_distance(ref, hyps[0], mode='word')
print('CER = {:.2f}%'.format(100. * res_cer['distance'] / res_cer['length']))
print('WER = {:.2f}%'.format(100. * res_wer['distance'] / res_wer['length']))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14928193499622072, 'fraction_numerical': 0.07407407407407407, 'mean_word_length': 4.448867536032944, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 22, 'https://': 12, 'lorem ipsum': 0, 'www.': 14, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '473927', 'license': 'MIT', 'n_tokens_mistral': 3482, 'n_tokens_neox': 3051, 'n_words': 626, 'path': 'example/train.ipynb', 'repo_name': 'paulhuangkm/MiniASR', 'size': '7946'}