<jupyter_start><jupyter_text>Feature ExtractionThis notebook will handle all feature extraction.This includes extracting the Mel-Frequency Cepstral Constant (MFCC), Zero Crossing Rate, Spectral Centroid,and Spectral Rolloff.The extracted data will be written to a formatted CSV file upon successful extraction.The produced CSV data file will then be utilised as training and testing data for the machine learning models.  Importing librariesAll utilised libraries are imported at the beginning of the notebook to avoid out of scope errors.<jupyter_code>import numpy as np
import librosa as lb
import librosa.display
import glob
from matplotlib import pyplot
import sklearn.preprocessing
import csv<jupyter_output><empty_output><jupyter_text>Function DefinitionsAll developer created functions are defined prior to the rest of the code to ensure accessibility.<jupyter_code># Function will loop through all files in the folder that utilise the .wav extension,
# extracting the time series data and appending it to an array
def extract_time_series(file_path):
    time_series_array = []
    for file_name in glob.glob(file_path + r'\*.wav'):
        (time_series_data, sample_rate) = lb.load(file_name)
    #   print(len(data)/sample_rate) -- Check tracks ingested correctly (result should be 30 seconds)
        time_series_data = np.array(time_series_data)
        time_series_array.append(time_series_data)
    return time_series_array

# Function will normalise data for plotting the spectral centroid
def norm(data, axis = 0):
    return sklearn.preprocessing.minmax_scale(data, axis=axis)<jupyter_output><empty_output><jupyter_text>Declare and Initialise VariablesVariables will be declared and initialised prior to use.The variables declared here are utilised throughout the notebook.<jupyter_code># ([] for i in range(0, 10, 1)) will create an empty array for
# the count of the range,which in this case is 10

rock_mfcc, metal_mfcc, country_mfcc, classical_mfcc, disco_mfcc, \
hiphop_mfcc, jazz_mfcc, reggae_mfcc, blues_mfcc, pop_mfcc = ([] for i in range(0, 10, 1))

rock_zcr, metal_zcr, country_zcr, classical_zcr, disco_zcr, \
hiphop_zcr, jazz_zcr, reggae_zcr, blues_zcr, pop_zcr  = ([] for i in range(0, 10, 1))

rock_spectral_centroid, metal_spectral_centroid, country_spectral_centroid, classical_spectral_centroid, \
disco_spectral_centroid,hiphop_spectral_centroid, jazz_spectral_centroid, reggae_spectral_centroid, \
blues_spectral_centroid, pop_spectral_centroid = ([] for i in range(0, 10, 1))

rock_spectral_rolloff, metal_spectral_rolloff, country_spectral_rolloff, classical_spectral_rolloff, \
disco_spectral_rolloff, hiphop_spectral_rolloff, jazz_spectral_rolloff, reggae_spectral_rolloff, \
blues_spectral_rolloff, pop_spectral_rolloff = ([] for i in range(0, 10, 1))<jupyter_output><empty_output><jupyter_text>Reading .wav filesDeclaring file paths for the corresponding directories for each genre.These paths are defined relative to the developers file structure.<jupyter_code>rock_file_path = "GTZAN/rock"
metal_file_path = "GTZAN/metal"
country_file_path = "GTZAN/country"
classical_file_path = "GTZAN/classical"
disco_file_path = "GTZAN/disco"
hiphop_file_path = "GTZAN/hiphop"
jazz_file_path = "GTZAN/jazz"
reggae_file_path = "GTZAN/reggae"
blues_file_path = "GTZAN/blues"
pop_file_path = "GTZAN/pop"<jupyter_output><empty_output><jupyter_text>Load .wav files using Librosa load and extract the time series data into an array for each track in each genre.The first value of each array is output as a check that values have been extracted successfully.<jupyter_code>print("Rock tracks:")
rock_tracks = extract_time_series(rock_file_path)
print(rock_tracks[0])

print("Metal tracks:")
metal_tracks = extract_time_series(metal_file_path)
print(metal_tracks[0])

print("Country tracks:")
country_tracks = extract_time_series(country_file_path)
print(country_tracks[0])

print("Classical tracks:")
classical_tracks = extract_time_series(classical_file_path)
print(classical_tracks[0])

print("Disco tracks:")
disco_tracks = extract_time_series(disco_file_path)
print(disco_tracks[0])

print("HipHop tracks:")
hiphop_tracks = extract_time_series(hiphop_file_path)
print(hiphop_tracks[0])

print("Jazz tracks:")
jazz_tracks = extract_time_series(jazz_file_path)
print(jazz_tracks[0])

print("Reggae tracks:")
reggae_tracks = extract_time_series(reggae_file_path)
print(reggae_tracks[0])

print("Blues tracks:")
blues_tracks = extract_time_series(blues_file_path)
print(blues_tracks[0])

print("Pop tracks:")
pop_tracks = extract_time_series(pop_file_path)
print(pop_tracks[0])<jupyter_output>Rock tracks:
[-0.03344727 -0.05490112 -0.05435181 ... -0.08416748  0.02886963
  0.1296997 ]
Metal tracks:
[-0.12237549 -0.04919434  0.04440308 ... -0.17837524 -0.08401489
 -0.0284729 ]
Country tracks:
[-0.13415527 -0.19485474 -0.04257202 ...  0.2260437   0.15042114
  0.04238892]
Classical tracks:
[-0.02008057 -0.01748657  0.00418091 ...  0.01934814  0.027771
  0.03131104]
Disco tracks:
[-0.23202515 -0.23168945 -0.1979065  ... -0.17419434 -0.14257812
 -0.10617065]
HipHop tracks:
[-0.16915894 -0.021698    0.01956177 ...  0.07662964  0.03872681
 -0.02105713]
Jazz tracks:
[-0.00924683 -0.01177979 -0.01370239 ...  0.0071106   0.00561523
  0.00561523]
Reggae tracks:
[ 0.01037598  0.00775146  0.03887939 ... -0.06652832 -0.05062866
 -0.03564453]
Blues tracks:
[ 0.00732422  0.01660156  0.00762939 ... -0.05560303 -0.06106567
 -0.06417847]
Pop tracks:
[-0.0887146  -0.09524536 -0.10275269 ...  0.04016113  0.03860474
  0.02639771]<jupyter_text>Feature extractionUsing Librosa feature extraction, extract the MFCC, Spectral Centroid, Spectral Rolloff,and Zero Crossings for each track in each genre.As each track is the same length, the total Zero Crossings has been utilised.  Extracting MFCCExtracting the Mel-frequency Cepstral Coefficients.The mean of 15 MFCC values calculated per track will be used as the MFCC for that track.<jupyter_code>for i in range(0, 100, 1):
    rock_mfcc.append(np.mean(lb.feature.mfcc(rock_tracks[i].flatten(), n_mfcc=15)))
    metal_mfcc.append(np.mean(lb.feature.mfcc(metal_tracks[i].flatten(), n_mfcc=15)))
    country_mfcc.append(np.mean(lb.feature.mfcc(country_tracks[i].flatten(), n_mfcc=15)))
    classical_mfcc.append(np.mean(lb.feature.mfcc(classical_tracks[i].flatten(), n_mfcc=15)))
    disco_mfcc.append(np.mean(lb.feature.mfcc(disco_tracks[i].flatten(), n_mfcc=15)))
    hiphop_mfcc.append(np.mean(lb.feature.mfcc(hiphop_tracks[i].flatten(), n_mfcc=15)))
    jazz_mfcc.append(np.mean(lb.feature.mfcc(jazz_tracks[i].flatten(), n_mfcc=15)))
    reggae_mfcc.append(np.mean(lb.feature.mfcc(reggae_tracks[i].flatten(), n_mfcc=15)))
    blues_mfcc.append(np.mean(lb.feature.mfcc(blues_tracks[i].flatten(), n_mfcc=15)))
    pop_mfcc.append(np.mean(lb.feature.mfcc(pop_tracks[i].flatten(), n_mfcc=15)))

print("Rock[0] MFCC:")
print(rock_mfcc[0])<jupyter_output>Rock[0] MFCC:
0.15879656<jupyter_text>Extracting ZCRExtracting the Zero Crossing Rate using Librosa's Zero Crossings function.The librosa.features.zero_crossing_rate behaves slightly differently to the behaviour wantedby the developer and will not be used for this reason.A single graphical output is used as a visual representation of the ZCR.<jupyter_code># The below code will produce a graph to illustrate zero crossings.
# This graph will be made utilising the first 100 frames of the first rock track.

pyplot.figure(figsize = (20, 5))
pyplot.plot(rock_tracks[0][0:100])
pyplot.grid()
crossing_val = sum(lb.zero_crossings(rock_tracks[0][0:100], pad=False))
print("Total zero crossings counted in the graph below:" + str(crossing_val))<jupyter_output>Total zero crossings counted in the graph below:10<jupyter_text>Counting the number of zero crossings throughout each 30 second track.zero_crossings returns an array of boolean values indicative of if a crossing has occurred.The sum of this array will return the count of true elements, indicative of the number of zero crossings.<jupyter_code># Setting pad to false means that a value of 0 is not considered a crossing.
# This has been set as the developer only wants to count actual crosses
# of the axis from positive to negative and vice versa

for i in range(0, 100, 1):
    rock_zcr.append(sum(lb.zero_crossings(rock_tracks[i], pad=False)))
    metal_zcr.append(sum(lb.zero_crossings(metal_tracks[i], pad=False)))
    country_zcr.append(sum(lb.zero_crossings(country_tracks[i], pad=False)))
    classical_zcr.append(sum(lb.zero_crossings(classical_tracks[i], pad=False)))
    disco_zcr.append(sum(lb.zero_crossings(disco_tracks[i], pad=False)))
    hiphop_zcr.append(sum(lb.zero_crossings(hiphop_tracks[i], pad=False)))
    jazz_zcr.append(sum(lb.zero_crossings(jazz_tracks[i], pad=False)))
    reggae_zcr.append(sum(lb.zero_crossings(reggae_tracks[i], pad=False)))
    blues_zcr.append(sum(lb.zero_crossings(blues_tracks[i], pad=False)))
    pop_zcr.append(sum(lb.zero_crossings(pop_tracks[i], pad=False)))

print("Rock[0] ZCR:")
print(rock_zcr[0])<jupyter_output>Rock[0] ZCR:
59809<jupyter_text>Extracting Spectral CentroidAs lb.feature.spectral_centroid() returns an array of values for each frame,the mean will be calculated for a value for the track.<jupyter_code>spec_cen = lb.feature.spectral_centroid(rock_tracks[0])[0]
frames = range(len(spec_cen))
time = lb.frames_to_time(frames)

lb.display.waveplot(rock_tracks[0], alpha=0.5)
pyplot.plot(time, norm(spec_cen), color='r')

for i in range(0, 100, 1):
    rock_spectral_centroid.append(np.mean(lb.feature.spectral_centroid(rock_tracks[i])))
    metal_spectral_centroid.append(np.mean(lb.feature.spectral_centroid(metal_tracks[i])))
    country_spectral_centroid.append(np.mean(lb.feature.spectral_centroid(country_tracks[i])))
    classical_spectral_centroid.append(np.mean(lb.feature.spectral_centroid(classical_tracks[i])))
    disco_spectral_centroid.append(np.mean(lb.feature.spectral_centroid(disco_tracks[i])))
    hiphop_spectral_centroid.append(np.mean(lb.feature.spectral_centroid(hiphop_tracks[i])))
    jazz_spectral_centroid.append(np.mean(lb.feature.spectral_centroid(jazz_tracks[i])))
    reggae_spectral_centroid.append(np.mean(lb.feature.spectral_centroid(reggae_tracks[i])))
    blues_spectral_centroid.append(np.mean(lb.feature.spectral_centroid(blues_tracks[i])))
    pop_spectral_centroid.append(np.mean(lb.feature.spectral_centroid(pop_tracks[i])))

print("Rock[0] Spectral Centroid:")
print(rock_spectral_centroid[0])<jupyter_output>Rock[0] Spectral Centroid:
1996.327664393569<jupyter_text>Extracting Spectral RolloffAs lb.feature.spectral_rolloff() returns an array of spectral rolloff % per a frame,the mean will be calculated to give an overall value per a track.<jupyter_code>for i in range(0, 100, 1):
    rock_spectral_rolloff.append(np.mean(lb.feature.spectral_rolloff(rock_tracks[i])))
    metal_spectral_rolloff.append(np.mean(lb.feature.spectral_rolloff(metal_tracks[i])))
    country_spectral_rolloff.append(np.mean(lb.feature.spectral_rolloff(country_tracks[i])))
    classical_spectral_rolloff.append(np.mean(lb.feature.spectral_rolloff(classical_tracks[i])))
    disco_spectral_rolloff.append(np.mean(lb.feature.spectral_rolloff(disco_tracks[i])))
    hiphop_spectral_rolloff.append(np.mean(lb.feature.spectral_rolloff(hiphop_tracks[i])))
    jazz_spectral_rolloff.append(np.mean(lb.feature.spectral_rolloff(jazz_tracks[i])))
    reggae_spectral_rolloff.append(np.mean(lb.feature.spectral_rolloff(reggae_tracks[i])))
    blues_spectral_rolloff.append(np.mean(lb.feature.spectral_rolloff(blues_tracks[i])))
    pop_spectral_rolloff.append(np.mean(lb.feature.spectral_rolloff(pop_tracks[i])))

print("Rock[0] Spectral Rolloff:")
print(rock_spectral_rolloff[0])<jupyter_output>Rock[0] Spectral Rolloff:
4227.935352015661<jupyter_text>Store DataWrite extracted data to a CSV file.<jupyter_code>with open("extracted-features.csv", mode='w', newline='') as feature_file:
    feature_writer = csv.writer(feature_file, delimiter=',')
    feature_writer.writerow(["Genre", "MFCC", "ZCR", "Spectral Centroid", "Spectral Rolloff"])

    for i in range(0, 1000, 1):
        if i < 100:
            track_num = i
            feature_writer.writerow(["Rock", rock_mfcc[track_num], rock_zcr[track_num],
                                    rock_spectral_centroid[track_num], rock_spectral_rolloff[track_num]])
        elif i < 200:
            track_num = i - 100
            feature_writer.writerow(["Metal", metal_mfcc[track_num], metal_zcr[track_num],
                                    metal_spectral_centroid[track_num], metal_spectral_rolloff[track_num]])
        elif i < 300:
            track_num = i - 200
            feature_writer.writerow(["Country", country_mfcc[track_num], country_zcr[track_num],
                                    country_spectral_centroid[track_num], country_spectral_rolloff[track_num]])
        elif i < 400:
            track_num = i - 300
            feature_writer.writerow(["Classical", classical_mfcc[track_num], classical_zcr[track_num],
                                    classical_spectral_centroid[track_num], classical_spectral_rolloff[track_num]])
        elif i < 500:
            track_num = i - 400
            feature_writer.writerow(["Disco", disco_mfcc[track_num], disco_zcr[track_num],
                                    disco_spectral_centroid[track_num], disco_spectral_rolloff[track_num]])
        elif i < 600:
            track_num = i - 500
            feature_writer.writerow(["HipHop", hiphop_mfcc[track_num], hiphop_zcr[track_num],
                                    hiphop_spectral_centroid[track_num], hiphop_spectral_rolloff[track_num]])
        elif i < 700:
            track_num = i - 600
            feature_writer.writerow(["Jazz", jazz_mfcc[track_num], jazz_zcr[track_num],
                                    jazz_spectral_centroid[track_num], jazz_spectral_rolloff[track_num]])
        elif i < 800:
            track_num = i - 700
            feature_writer.writerow(["Reggae", reggae_mfcc[track_num], reggae_zcr[track_num],
                                     reggae_spectral_centroid[track_num], reggae_spectral_rolloff[track_num]])
        elif i < 900:
            track_num = i - 800
            feature_writer.writerow(["Blues", blues_mfcc[track_num], blues_zcr[track_num],
                                    blues_spectral_centroid[track_num], blues_spectral_rolloff[track_num]])
        elif i < 1000:
            track_num = i - 900
            feature_writer.writerow(["Pop", pop_mfcc[track_num], pop_zcr[track_num],
                                    pop_spectral_centroid[track_num], pop_spectral_rolloff[track_num]])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10645534317393375, 'fraction_numerical': 0.05135705054077954, 'mean_word_length': 5.0106295993458705, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 39, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '574964', 'license': 'MIT', 'n_tokens_mistral': 5633, 'n_tokens_neox': 5055, 'n_words': 1002, 'path': 'feature-extraction.ipynb', 'repo_name': 'JackW-Dev/Music-Genre-Classification', 'size': '14701'}