<jupyter_start><jupyter_text>Sound classification with YAMNetThis is followed from Tensorflow's official docs and the link for this doc is [this](https://www.tensorflow.org/hub/tutorials/yamnet).YAMNet is a deep net that predicts 521 audio event [classes](https://github.com/tensorflow/models/blob/master/research/audioset/yamnet/yamnet_class_map.csv) from the [AudioSet-YouTube corpus](https://research.google.com/audioset/) it was trained on. It employs the [Mobilenet_v1](https://arxiv.org/pdf/1704.04861.pdf) depthwise-separable convolution architecture.<jupyter_code>import csv

import numpy as np
import matplotlib.pyplot as plt

from scipy.io import wavfile

import tensorflow as tf
import tensorflow_hub as hub

from IPython.display import Audio<jupyter_output><empty_output><jupyter_text>Load the Model from TensorFlow Hub.<jupyter_code># Load the model.
model = hub.load('https://tfhub.dev/google/yamnet/1')<jupyter_output><empty_output><jupyter_text>The labels file will be loaded from the models assets and is present at `model.class_map_path()`. You will load it on the `class_names` variable.<jupyter_code># Find the name of the class with the top score when mean-aggregated across frames.
def class_names_from_csv(class_map_csv_text):
    ''' Returns list of class names corresponding to score vector. '''
    
    class_names = []
    with tf.io.gfile.GFile(class_map_csv_text) as csv_file:
        reader = csv.DictReader(csv_file)
        for row in reader:
            class_names.append(row['display_name'])
    return class_names


class_map_path = model.class_map_path().numpy()
class_names = class_names_from_csv(class_map_path)<jupyter_output><empty_output><jupyter_text>Add a method to verify and convert a loaded audio is on the proper sample_rate (16K), otherwise it would affect the model's results.<jupyter_code>def ensure_sample_rate(original_sample_rate, waveform, desired_sample_rate=16000):
    ''' Resample waveform if required. '''
    
    if original_sample_rate != desired_sample_rate:
        desired_length = int(round(float(len(waveform)) / original_sample_rate * desired_sample_rate))
        waveform = scipy.signal.resample(waveform, desired_length)
    return desired_sample_rate, waveform<jupyter_output><empty_output><jupyter_text>Downloading and preparing the sound fileHere you will download a wav file and listen to it. If you have a file already available, just upload it to colab and use it instead.**Note**: The expected audio file should be a mono wav file at 16kHz sample rate.<jupyter_code>!curl -O https://storage.googleapis.com/audioset/speech_whistling2.wav
!curl -O https://storage.googleapis.com/audioset/miaow_16k.wav
# wav_file_name = 'speech_whistling2.wav'
wav_file_name = 'miaow_16k.wav'
sample_rate, wav_data = wavfile.read(wav_file_name, 'rb')
sample_rate, wav_data = ensure_sample_rate(sample_rate, wav_data)

# Show some basic information about the audio.
duration = len(wav_data)/sample_rate
print(f'Sample rate: {sample_rate} Hz')
print(f'Total duration: {duration:.2f}s')
print(f'Size of the input: {len(wav_data)}')

# Listening to the wav file.
Audio(wav_data, rate=sample_rate)<jupyter_output>Sample rate: 16000 Hz
Total duration: 6.73s
Size of the input: 107698<jupyter_text>The `wav_data` needs to be normalized to values in `[-1.0, 1.0]` (as stated in the model's [documentation](https://tfhub.dev/google/yamnet/1)).<jupyter_code>waveform = wav_data / tf.int16.max<jupyter_output><empty_output><jupyter_text>Executing the ModelNow the easy part: using the data already prepared, you just call the model and get the: scores, embedding and the spectrogram.The score is the main result you will use. The spectrogram you will use to do some visualizations later.<jupyter_code># Run the model, check the output.
scores, embeddings, spectrogram = model(waveform)
scores_np = scores.numpy()
spectrogram_np = spectrogram.numpy()
infered_class = class_names[scores_np.mean(axis=0).argmax()]
print(f'The main sound is: {infered_class}')<jupyter_output>The main sound is: Animal<jupyter_text>VisualizationYAMNet also returns some additional information that we can use for visualization. Let's take a look on the Waveform, spectrogram and the top classes inferred.<jupyter_code>plt.figure(figsize=(10, 6))

# Plot the waveform.
plt.subplot(3, 1, 1)
plt.plot(waveform)
plt.xlim([0, len(waveform)])

# Plot the log-mel spectrogram (returned by the model).
plt.subplot(3, 1, 2)
plt.imshow(spectrogram_np.T, aspect='auto', interpolation='nearest', origin='lower')

# Plot and label the model output scores for the top-scoring classes.
mean_scores = np.mean(scores, axis=0)
top_n = 10
top_class_indices = np.argsort(mean_scores)[::-1][:top_n]
plt.subplot(3, 1, 3)
plt.imshow(scores_np[:, top_class_indices].T, aspect='auto', interpolation='nearest', cmap='gray_r')

# patch_padding = (PATCH_WINDOW_SECONDS / 2) / PATCH_HOP_SECONDS
# values from the model documentation
patch_padding = (0.025 / 2) / 0.01
plt.xlim([-patch_padding-0.5, scores.shape[0] + patch_padding-0.5])
# Label the top_N classes.
yticks = range(0, top_n, 1)
plt.yticks(yticks, [class_names[top_class_indices[x]] for x in yticks])
_ = plt.ylim(-0.5 + np.array([top_n, 0]))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10586661570800687, 'fraction_numerical': 0.016816357729791707, 'mean_word_length': 5.7535483870967745, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 8, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '159896', 'license': 'Apache-2.0', 'n_tokens_mistral': 1856, 'n_tokens_neox': 1749, 'n_words': 478, 'path': 'notebooks/sound-classification-with-yamnet.ipynb', 'repo_name': 'AkashSDas/tensorflow-for-audio-101', 'size': '5233'}