<jupyter_start><jupyter_text>Welcome to ExKaldiIn this section, we will extract and process the acoustic feature.Please ensure you have downloaded the complete librispeech_dummy corpus from our github.https://github.com/wangyu09/exkaldiFirst of all, update the wav path info in wav.scp file.<jupyter_code>! cd librispeech_dummy && python3 reset_wav_path.py<jupyter_output><empty_output><jupyter_text>From now on, we will start to build a ASR system from the scratch.<jupyter_code>import exkaldi

import os
dataDir = "librispeech_dummy"<jupyter_output><empty_output><jupyter_text>In the train dataset, there are 100 utterances fetched from 10 speakers. Each specker corresponds to 10 utterances.You can compute feature from __WAV file__ or __Kaldi script-file table__ or exkaldi __ListTable__ object.<jupyter_code>scpFile = os.path.join(dataDir, "train", "wav.scp")

feat = exkaldi.compute_mfcc(scpFile, name="mfcc")

feat<jupyter_output><empty_output><jupyter_text>Use function __compute_mfcc__ to compute MFCC feature. In current version of ExKaldi, there are 4 functions to compute acoustic feature:__compute_mfcc__: compute the MFCC feature.  __compute_fbank__: compute the fBank feature.  __compute_plp__: compute the PLP feature.  __compute_spectrogram__: compute the power spectrogram feature.  The returned object: ___feat___ is an exkaldi feature archive whose class name is __BytesFeat__. In ExKaldi, we use 3 approaches to discribe Kaldi archives: __Bytes Object__, __Numpy Array__, and __Index Table__. We have designed a group of classes to hold them. We will introduce them in later steps.Here, __BytesFeat__ is one of __Bytes Object__ and its object holds the acoustic feature data with bytes format. You can use attribute: __.data__ to get it, but we do not recommend this if you just want to look it, because it is not a human-readable data format.___feat___ object has some useful attributes and methods. For example, use __.dim__ to look feature dimensions.<jupyter_code>feat.dim<jupyter_output><empty_output><jupyter_text>Use __.utts__ to get its' utterances IDs.<jupyter_code>feat.utts[0:5]<jupyter_output><empty_output><jupyter_text>Randomly sample 10 utterances.<jupyter_code>samplingFeat = feat.subset(nRandom=10)

samplingFeat<jupyter_output><empty_output><jupyter_text>Here, ___samplingFeat___ is also a __BytesFeat__ object.In ExKaldi, the name of object will record the operation. For example, the ___samplingFeat___ generated above has a new name now.<jupyter_code>samplingFeat.name
del samplingFeat<jupyter_output><empty_output><jupyter_text>Besides __BytesFeat__ class, these classes can hold other Kaldi archive tables in bytes format.__BytesCMVN__: to hold the CMVN statistics.  __BytesProb__: to hold the Neural Network output.  __BytesAliTrans__: to hold the transition-ID Alignment.   __BytesFmllr__: to hold the fmllr transform matrices. All these classes have some fimiliar properties. For more information, check the [ExKaldi Documents](https://wangyu09.github.io/exkaldi//) please. Here we only focus on feature processing.By the way, in ExKaldi, we sort these archives rigorously in order to reduce buffer cost and accelerate processing.<jupyter_code>featTemp = feat.sort(by="utt", reverse=True)

featTemp.utts[0:5]
del featTemp<jupyter_output><empty_output><jupyter_text>Raw feature can be further optimized, typically, with applying CMVN. Here we firstly compute the CMVN statistics.<jupyter_code>spk2uttFile = os.path.join(dataDir, "train", "spk2utt")

cmvn = exkaldi.compute_cmvn_stats(feat, spk2utt=spk2uttFile, name="cmvn")

cmvn<jupyter_output><empty_output><jupyter_text>___cmvn___ is an exkaldi __BytesCMVN__ object. It holds the CMVN statistics in binary format. Then we use it to normalize the feature.<jupyter_code>utt2spkFile = os.path.join(dataDir, "train", "utt2spk")

feat = exkaldi.use_cmvn(feat, cmvn, utt2spk=utt2spkFile)

feat.name<jupyter_output><empty_output><jupyter_text>We save this feature into file. In futher steps, it will be restoraged. ExKaldi bytes archives can be saved the same as Kaldi format files.<jupyter_code>featFile = os.path.join(dataDir, "exp", "train_mfcc_cmvn.ark")

exkaldi.utils.make_dependent_dirs(path=featFile, pathIsFile=True)

featIndex = feat.save(featFile, returnIndexTable=True)

#del feat<jupyter_output><empty_output><jupyter_text>If you appoint the option __returnIndexTable__ to be True, an __IndexTable__ object will be returned. As we introduced above, this is our second approach to discribe archives, __index table__. It plays almost the same role with original feature object. __IndexTable__ is a subclass of Python dict class, so you can view its data directly.When training a large corpus or using multiple processes, __IndexTable__ will become the main currency.<jupyter_code>featIndex<jupyter_output><empty_output><jupyter_text>Of cause, original archives can also be loaded into memory again. For example, feature can be loaded from Kaldi binary archive file (__.ark__ file) or script table file (__.scp__).Particularly, we can fetch the data via index table directly.<jupyter_code>feat = featIndex.fetch(arkType="feat")
del featIndex

feat<jupyter_output><empty_output><jupyter_text>All Bytes archives can be transformed to __Numpy__ format. So If you want to train NN acoustic model with Tensorflow or others, you can use the Numpy format data.<jupyter_code>feat = feat.to_numpy()

feat<jupyter_output><empty_output><jupyter_text>by calling __.to_numpy()__ function, ___feat___ became an exkaldi __NumpyFeat__ object, it has some fimiliar attributes and methods with __BytesFeat__, but also has own properties. Let's skip the details here.This is the third way to discribe archives: __Numpy Array__. __NumpyFeat__ is one of Numpy archives classes.Here we will introduce some methods to use its data.<jupyter_code>sampleFeat = feat.subset(nHead=2)<jupyter_output><empty_output><jupyter_text>1. use __.data__ to get the dict object whose keys are utterance IDs and values are data arrays.<jupyter_code>sampleFeat.data<jupyter_output><empty_output><jupyter_text>2. use __.array__ get the arrays only.<jupyter_code>sampleFeat.array<jupyter_output><empty_output><jupyter_text>3. use getitem function to get a specified utterance.<jupyter_code>sampleFeat['103-1240-0000']<jupyter_output><empty_output><jupyter_text>4. like dict object, __.keys()__,__.values()__,__.items()__ are availabel to iterate it.<jupyter_code>for key in sampleFeat.keys():
    print( sampleFeat[key].shape )<jupyter_output><empty_output><jupyter_text>5. setitem is also available only if you set the array with right format.<jupyter_code>sampleFeat['103-1240-0000'] *= 2
sampleFeat['103-1240-0000']
del sampleFeat<jupyter_output><empty_output><jupyter_text>Similarly, ExKaldi Numpy archives can be transformed back to bytes archives easily.<jupyter_code>tempFeat = feat.to_bytes()

tempFeat
del tempFeat<jupyter_output><empty_output><jupyter_text>Numpy data can also be saved to .npy file with a specified format.<jupyter_code>tempFile = os.path.join(dataDir, "exp", "temp_mfcc.npy")

feat.save(tempFile)
del feat<jupyter_output><empty_output><jupyter_text>And it can also be restorage into memory again.<jupyter_code>feat = exkaldi.load_feat(tempFile, name="mfcc")

feat
feat<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07494852436513384, 'fraction_numerical': 0.009608785175017159, 'mean_word_length': 6.751063829787234, 'pattern_counts': {'":': 0, '<': 93, '<?xml version=': 0, '>': 93, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '52206', 'license': 'Apache-2.0', 'n_tokens_mistral': 2408, 'n_tokens_neox': 2278, 'n_words': 751, 'path': 'tutorials/02_feature_processing.ipynb', 'repo_name': 'wangyu09/exkaldi', 'size': '7285'}