<jupyter_start><jupyter_text>To train up a Model with the .wav dataset. Run through this Notebook<jupyter_code>#Download Dataset
!wget https://zenodo.org/record/4682101/files/dataset.zip
#Unzip Dataset
!unzip dataset.zip > 0
# import Required Libraries
from matplotlib import cm
import os.path
import random
import pathlib
import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
import tensorflow.compat.v1.keras.backend as K
from feature_extract import *
from train import Model
from tflite_op import *
from feature_extract import *

if tf.test.is_built_with_cuda():
    from tensorflow.compat.v1 import ConfigProto
    from tensorflow.compat.v1 import InteractiveSession
    config = ConfigProto()
    config.gpu_options.allow_growth = True
    sess = InteractiveSession(config=config)
else:
    sess = InteractiveSession()
#!unzip dataset.zip > 0
""" 
    Read Samples directory into an Array Excluding irrelivant directories
"""
data_dir = pathlib.Path('dataset')

commands = np.array(tf.io.gfile.listdir(str(data_dir)))
commands = commands[commands != 'validation_list.txt']
commands = commands[commands != "_background_noise_"]
commands = commands[commands != 'LICENSE']
commands = commands[commands != '.DS_Store']
commands = commands[commands != 'README.md']
commands = commands[commands != 'testing_list.txt']
commands = commands[commands != '.ipynb_checkpoints']
commands = commands[commands != 'silence']
commands = list(commands)
print("Number of Commands: ", len(commands))
!rm -r dataset/_background_noise_
!rm -r dataset/silence
model_dir = pathlib.Path('checkpoints')
search_path = os.path.join(data_dir, "*", '*.wav') # Collect all available ./wav files into a python list
files = gfile.Glob(search_path)
np.random.shuffle(files)
# Divide file samples into training, Validation and Testing

n_of_samples = len(files)
train_percent = int(n_of_samples * 0.7) # Percentage of Data available for Training
val_percent = int((n_of_samples - train_percent)*0.7) # Percentage of Data available for Validation
test_percent = int((n_of_samples - train_percent)*0.3) # Percentage of Data available for Testing

train_files = files[:train_percent]
val_files = files[train_percent:val_percent+train_percent]
test_files= files[val_percent+train_percent:]

print("training samples: ", len(train_files))
print("validation Samples: ", len(val_files))
print("test Samples: ", len(test_files))<jupyter_output>training samples:  75098
validation Samples:  22529
test Samples:  9656<jupyter_text>Create Model and Model Layers. You can increase the CNN filters to desired values to increase accuracy<jupyter_code>model = Model(first_conv_filter = 128, second_conv_filter = 64, model_dir="checkpath", commands= commands, sess = sess) # Create Model
#Comment out to load from checkpoint, you can also pass in path to checkpoint
#model.load_checkpoint()
#model.load_checkpoint(path="model/model_checkpoint.ckpt-10000") 
# Train Model
history = model.train(learn_rate=[0.01, 0.0001], dropout_rate=0.5, save_step=1000, eval_step=1000,
                      batch_size=256, training_time=15000, rate_step=10000, display_step=1000, 
                      train_data=train_files, Validation_data=val_files)
plt.subplot(211)
plt.plot(history['categorical_accuracy'])
plt.plot(history['val_categorical_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train','val'],loc='upper left')# summarize history for loss 



plt.subplot(212)
plt.plot(history['loss'])
plt.plot(history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train','val'],loc='upper left')
plt.tight_layout()
plt.show()
data, label = model.get_next_batch(2000, file_path = test_files) # Grab 2000 files from Test files for Evaluation
model.evaluate(data, label) # Evaluate on Data # Please note<jupyter_output>Validation accuracy = 87.7%, Validation loss = 0.439979<jupyter_text>Let us See the Accuracy on Test Samples<jupyter_code>grab = int(len(test_files)/5)
fro = 0
correct = 0
for i in range(5):
    data, label = model.get_next_batch(int(len(test_files)/5), test_files[fro:grab])
    predictions = model.predict(data)[0]
    correct+=np.sum(predictions == label)
    fro = grab
    grab+=int(len(test_files)/5)
print(' Model accuracy is %f%% (Number of test samples=%d)' % (
        (correct * 100) / len(test_files), len(test_files)))<jupyter_output>Model accuracy is 87.572494% (Number of test samples=9656)<jupyter_text>Here we are Saving the Model for a reload using tf.saved_model.load, Serving Signature is set to Default<jupyter_code>model.save_pb_model(file_name = "Model/model_pb",  first_conv_filter=128, second_conv_filter=64, frequency_size=40, time_size=49, )<jupyter_output>INFO:tensorflow:Restoring parameters from checkpath/model_checkpoint.ckpt-15000
WARNING:tensorflow:From /home/jovyan/train.py:434: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
INFO:tensorflow:No assets to save.
INFO:tensorflow:No assets to write.
INFO:tensorflow:SavedModel written to: Model/model_pb/saved_model.pb<jupyter_text>The Next Lines of code introduces you to TensorFlow LITE (TFLITE) and Tensorflow LITE MICRO (TFLITE MICRO) TFLITE for Android Devices and Micro-Computers TFLITE Micro for Micro-controllers and Embedded Devices with low power and generally requiring int8 computations<jupyter_code># directories and tflite micro libraries
from tflite_op import *
rep_size = 100
float_model_tflite="Model/tfFloat"
quant_model_tflite= "Model/tflite"
pb_model_dir = "Model/model_pb"
tflite_micro = "Model/tflite_micro.cc"
# See tflite_op for implementation. Save TFLITE MODEL FOR FLOAT VALUE COMPUTATION
save_float_model(model_path=pb_model_dir, save_path=float_model_tflite)
# # See tflite_op for implementation. Save TFLITE MODEL FOR INT8 VALUE COMPUTATION
save_quantized_model(model_path=pb_model_dir, save_path=quant_model_tflite, rep_files=test_files, rep_data=100)<jupyter_output>WARNING:absl:Invoking the TF1 implementation of TFLiteConverter because eager is disabled. Consider enabling eager.<jupyter_text>Now Let us Test the Accuracy from TFLITE... See Implmentation in tflite_op.py<jupyter_code># Use our data from our wanted commands
commands_files =[]
for com in ["on", "off", "heater", "fan", "light"]:
    search_path = os.path.join("dataset", com, '*.wav')
    file = gfile.Glob(search_path)
    np.random.shuffle(file)
    commands_files += file
# Preprocess using the Micro_process Implementation
data, labels = model.get_next_batch(len(commands_files), commands_files)<jupyter_output><empty_output><jupyter_text>Great Result, 93% on our Wanted words with TFLITE saved as Float: See Result below<jupyter_code>correct_predictions = 0
for i in range(len(data)):
    correct_predictions += (predict_float(data[i], float_model_tflite) == labels[i])
   

print(' Float model accuracy is %f%% (Number of test samples=%d)' % (
        (correct_predictions * 100) / len(data), len(data)))<jupyter_output>Float model accuracy is 93.984962% (Number of test samples=9044)<jupyter_text>Great Result, 93% on our Wanted words with TFLITE saved as INT8: See Result below<jupyter_code>correct_predictions = 0
for i in range(len(data)):
    correct_predictions += (predict_quantized(data[i], quant_model_tflite ) == labels[i])
   

print(' Int8 model accuracy is %f%% (Number of test samples=%d)' % (
        (correct_predictions * 100) / len(data), len(data)))<jupyter_output>Int8 model accuracy is 93.841221% (Number of test samples=9044)<jupyter_text>For Embedded Devices, Let us covert to TFLITE Micro<jupyter_code>!xxd -i {quant_model_tflite} > {tflite_micro}
REPLACE_TEXT = quant_model_tflite.replace('/', '_').replace('.', '_')
!sed -i 's/'{REPLACE_TEXT}'/g_model/g' {tflite_micro}<jupyter_output><empty_output><jupyter_text>Holla!!! You have Just Created a MODEL that can be deployed for Speech Recognition on Embedded Devices. Next is to locate the file Model/tflite_micro.cc and this is the Model for deployment to our Device<jupyter_code># You can also copy the Parameters by running this Cell
# !cat {MODEL_TFLITE_MICRO}  
!zip model.zip -r Model
!zip checkpoint.zip -r checkpath 
!unzip check.zip -d work<jupyter_output>Archive:  check.zip
   creating: work/model/
  inflating: work/model/model.pbtxt  
  inflating: work/model/model_checkpoint.ckpt-7000.index  
  inflating: work/model/model_checkpoint.ckpt-8000.meta  
  inflating: work/model/model_checkpoint.ckpt-9000.meta  
  inflating: work/model/model_checkpoint.ckpt-9000.index  
  inflating: work/model/model_checkpoint.ckpt-8000.data-00000-of-00001  
  inflating: work/model/model_checkpoint.ckpt-7000.data-00000-of-00001  
  inflating: work/model/model_checkpoint.ckpt-8000.index  
  inflating: work/model/checkpoint   
  inflating: work/model/model_checkpoint.ckpt-9000.data-00000-of-00001  
  inflating: work/model/model_checkpoint.ckpt-10000.data-00000-of-00001  
  inflating: work/model/model_checkpoint.ckpt-6000.meta  
  inflating: work/model/model_checkpoint.ckpt-6000.data-00000-of-00001  
  inflating: work/model/model_checkpoint.ckpt-7000.meta  
  inflating: work/model/model_checkpoint.ckpt-6000.index  
  inflating: work/model/model_checkpoint.ckpt[...]
{'fraction_non_alphanumeric': 0.10745197382309478, 'fraction_numerical': 0.03103229892336922, 'mean_word_length': 5.865942028985507, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 36, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '45265', 'license': 'MIT', 'n_tokens_mistral': 3368, 'n_tokens_neox': 3082, 'n_words': 773, 'path': 'Large_Model/Training.ipynb', 'repo_name': 'Predstan/s21-team3-project', 'size': '9474'}