<jupyter_start><jupyter_text>Music genre classification notebookThis is the notebook with all the technical details of our work.  Libraries<jupyter_code>%load_ext autoreload
%autoreload 2
%reload_ext autoreload

# feature extractoring and preprocessing data
import librosa
import numpy as np
import pandas as pd

from server.utils import Util
from server.feature_extractor import FeatureAggregator

# Utils
import matplotlib.pyplot as plt
from tqdm import tqdm
from concurrent.futures import ThreadPoolExecutor

# model selecting and evaluation
from sklearn.preprocessing import LabelEncoder, StandardScaler
from sklearn.model_selection import train_test_split, GridSearchCV, StratifiedKFold
from sklearn.metrics import classification_report

# models
from sklearn.neighbors import KNeighborsClassifier
from sklearn.dummy import DummyClassifier
from sklearn.svm import SVC
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import VotingClassifier
import xgboost as xgb

import warnings
warnings.filterwarnings('ignore')<jupyter_output>The autoreload extension is already loaded. To reload it, use:
  %reload_ext autoreload<jupyter_text>Extracting music and featuresWe use [GTZAN genre collection](http://marsyasweb.appspot.com/download/data_sets/) dataset for classification. We classify * Classical  * Metal * Blues * Hiphop * Pop * Rock * Country * Reggae  * Jazz Each genre contains 100 songs. Total dataset: 900 songs<jupyter_code>genres=['classical', 'metal', 'blues', 'hiphop', 'pop', 'rock', 'country', 'reggae', 'jazz']
def get_data(genre, range_tuple=100):
    data = list()
    for i in range(range_tuple):
        if i < 10:
            path = "../../" + genre + "/" + genre + ".0000" + str(i) + ".au"        
        else:
            path = "../../" + genre + "/" + genre + ".000" + str(i) + ".au"
        song = librosa.load(path)
        data.append(song[0])
        if i%30 == 0: print("Got {0} songs for genre {1}".format(i, genre))
    return (data, genre)
with ThreadPoolExecutor(len(genres)) as pool:
    results = pool.map(get_data, genres)
results = list(results)<jupyter_output>Got 0 songs for genre reggae
Got 0 songs for genre pop
Got 0 songs for genre classical
Got 0 songs for genre jazz
Got 0 songs for genre hiphop
Got 0 songs for genre rock
Got 0 songs for genre blues
Got 0 songs for genre metal
Got 0 songs for genre country
Got 30 songs for genre reggae
Got 30 songs for genre pop
Got 30 songs for genre metal
Got 30 songs for genre jazz
Got 30 songs for genre country
Got 30 songs for genre classical
Got 30 songs for genre hiphop
Got 30 songs for genre rock
Got 30 songs for genre blues
Got 60 songs for genre reggae
Got 60 songs for genre pop
Got 60 songs for genre classical
Got 60 songs for genre metal
Got 60 songs for genre country
Got 60 songs for genre jazz
Got 60 songs for genre hiphop
Got 60 songs for genre blues
Got 60 songs for genre rock
Got 90 songs for genre pop
Got 90 songs for genre reggae
Got 90 songs for genre classical
Got 90 songs for genre country
Got 90 songs for genre metal
Got 90 songs for genre jazz
Got 90 songs for genre hiphop
Got 90[...]<jupyter_text>Extracting FeaturesWe will extract * Mel-frequency cepstral coefficients (MFCC). You can read about them here:     * [Wikipedia](https://en.wikipedia.org/wiki/Mel-frequency_cepstrum)     * [<NAME>. *Spectrogram, Cepstrum and Mel-Frequency Analysis.*](http://www.speech.cs.cmu.edu/15-492/slides/03_mfcc.pdf)     * [habr (article in Russian)](https://habr.com/post/140828/)     * [<NAME>. *Mel frequency spectral coefficients for Music Modeling*](http://musicweb.ucsd.edu/~sdubnov/CATbox/Reader/logan00mel.pdf) * Centoids. Measure of Spectral Brightness $$     C_t = \frac{\sum_{i=1}^N f_i M_t[f_i]}{\sum_{i=1}^N M_t[f_i]} $$ * Rolloff. Rate of Spectral Decreasing $$     R_t: \sum_{i=1}^{R_t}M_t[f_i] = 0.85 \cdot \sum_{i=1}^N M_t[f_i] $$ * Flux. Measure of Spectral Change $$     F_t = \|M_t[f] - M_{t-1}[f] \| $$ * Chromagram. Read about it here     * [Wikipedia](https://en.wikipedia.org/wiki/Chroma_feature) * Low Energy. Percentage of windows with less than average energy * Rhythm fearures     * bmp     * Tempogram     * autocorrelation * Zero Crossing Rate. Detecting noise in the signal $$     zcr = \frac{1}{T-1}\sum_{i=1} \textbf{1}_{R<0}(S_t S_{t-1}) $$  For vector features we count  * Mean (MFCC, Chroma, Centroids, Flux, Rolloff, ZRC, ect.) * Median (Centroids, Flux, Rolloff, ZRC, ect.) * Standard Deviation (Det[cov_matrix MFCC], Centroids, Flux, Rolloff, ZRC, ect.) * Max (Centroids, Flux, Rolloff, ZRC, ect.) * Min (Centroids, Flux, Rolloff, ZRC, ect.)<jupyter_code>music_list = list()
genre_list = list()
for genre_data in results: 
    music_list += genre_data[0]
    genre_list += list(np.array([genre_data[1]]).repeat(100))
exctractor = FeatureAggregator(music_list, parallel=True)
extracted, feature_names = exctractor.get_features()
# save all the extracted features to the csv file extracted_data.csv
data_to_save = np.hstack((extracted, np.array(genre_list).reshape(-1,1)))
pd.DataFrame(data_to_save, columns=feature_names + ['genre']).to_csv("extracted_data.csv", encoding="utf-8")<jupyter_output><empty_output><jupyter_text>Data ScienceHere is some brief information about the code below. Models declaration**Bad results**:  * DummyClassifier (obviously bad, just to compare) **Satisfactory results** * KNeighborsClassifier  * xgboost**Good results**  * SVC * RandomForestClassifier * VotingClassifier: ensemle of the previous 2 with soft voting How do we fit parametersGridSearchCV How do we evaluate modelsWe iterate randoms_state in train_test_split with the future averaging of the classification reports<jupyter_code># Iterating random_state in train_test_split with future averaging of the classification reports
def check_model(model, X, y, encoder, n=10):
    num_genres = len(np.unique(y))
    average = np.zeros((num_genres + 1, 4), dtype = float)
    for rs in tqdm(range(n)):
        X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=rs, stratify=y)
        model.fit(X_train, y_train)
        target = encoder.inverse_transform(np.arange(num_genres))
        df = parse_class_report(classification_report(y_test, model.predict(X_test), target_names=target))
        average += df.values
    df.iloc[:,:] = np.round(average / n, 3)
    return df

# Iterating random_state in train_test_split with future averaging of the classification reports for xgboost
def check_model_xgb(X, y, encoder, n=10):
    num_genres = len(np.unique(y))
    average = np.zeros((num_genres + 1, 4), dtype = float)
    for rs in tqdm(range(n)):
        X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=rs, stratify=y)
        dtrain = xgb.DMatrix(X_train, label=y_train)
        dtest = xgb.DMatrix(X_test, label=y_test)
        param = {'objective': "multi:softmax", "num_class": np.unique(y_train).size}
        evallist = [(dtrain, 'train'), (dtest, 'eval')]
        bst = xgb.train(param, dtrain, evals=evallist, verbose_eval=False)
        target = encoder.inverse_transform(np.arange(num_genres))
        df = parse_class_report(classification_report(y_test, bst.predict(dtest), target_names=target))
        average += df.values
    df.iloc[:,:] = np.round(average / n, 3)
    return df

# Transforming classification_report string to pandas.DataFrame
def parse_class_report(class_rep_str):
    lines = class_rep_str.split('\n')
    list_splitted = list()
    for line in lines:
        splitted_line = np.array(line.split(' '))
        mask_non_empty = np.array([word != '' for word in line.split(' ')])
        res = splitted_line[mask_non_empty]
        if len(res) != 0:
            list_splitted.append(np.array(res, dtype = object))

    first = list(list_splitted[0])
    first.insert(0, 'genre')
    list_splitted[0] = np.array(first, dtype = object)
    list_splitted[-1] = list_splitted[-1][2:]
    list_splitted[-1][0] = 'avg / total'
    
    df = pd.DataFrame(list_splitted)
    df.columns = list(df.iloc[0])
    return df.iloc[1:].set_index('genre').convert_objects(convert_numeric=True)<jupyter_output><empty_output><jupyter_text>Models Preparing data<jupyter_code>df = pd.DataFrame.from_csv("extracted_data.csv", encoding = 'utf-8')

scaler = StandardScaler()
X = scaler.fit_transform(np.array(df.values[:, :-1], dtype = float))

genre_list = df.values[:, -1]
encoder = LabelEncoder()
y = encoder.fit_transform(genre_list)<jupyter_output><empty_output><jupyter_text>GridSearchCV and final preparation<jupyter_code>dummy = DummyClassifier()
knn = KNeighborsClassifier()
rfc = RandomForestClassifier()
svc = SVC(probability=True)

param_grid_rfc = { 
    'n_estimators': [ 70, 150, 370],
    'max_features': ['log2'],
    'max_depth' : [10,15, 20],
    'criterion' :['gini']
}

param_grid_svc = {
    "C": np.logspace(0,2,num=20), 
    "kernel": ["poly", "rbf", "sigmoid"]   
}

param_grid_knn = {
    'n_neighbors' : [2,5,8,10,15]
}

skf = StratifiedKFold(n_splits=3)

grid_rfc = GridSearchCV(estimator=rfc, param_grid=param_grid_rfc, cv=skf, n_jobs = -1)
grid_svc = GridSearchCV(estimator=svc, param_grid=param_grid_svc, cv=skf, n_jobs = -1)
grid_knn = GridSearchCV(estimator=knn, param_grid=param_grid_knn, cv=skf, n_jobs = -1)

eclf = VotingClassifier(estimators=[('svc', grid_svc), ('rfc', grid_rfc)], voting='soft')
# DummyClassifier
check_model(dummy, X, y, encoder)
# KNeighbors
check_model(grid_knn, X, y, encoder)
# xgboost
check_model_xgb(X, y, encoder, n=10)
# RandomForest
check_model(grid_rfc, X, y, encoder)
# Support Vectors
check_model(grid_svc, X, y, encoder)
# Voting Classifier
check_model(eclf, X, y, encoder)<jupyter_output>100%|██████████| 10/10 [03:20<00:00, 21.00s/it]
{'fraction_non_alphanumeric': 0.11858842839556832, 'fraction_numerical': 0.02082478457119409, 'mean_word_length': 4.596440872560276, 'pattern_counts': {'":': 3, '<': 27, '<?xml version=': 0, '>': 24, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '355473', 'license': 'MIT', 'n_tokens_mistral': 3566, 'n_tokens_neox': 3295, 'n_words': 922, 'path': 'Music genre classification.ipynb', 'repo_name': 'minemile/music_genre_classification', 'size': '9763'}