<jupyter_start><jupyter_text>Notebook for the Analysis of Audio Data---This notebook perfroms a simple audio analysis based around the [Fast Fourier Transformation](https://en.wikipedia.org/wiki/Fast_Fourier_transform) of wave audio files (wav).For the import and transformation of the wav files different features from the scipy package are used.Finally a classification of the files is made using the Extra Trees Classifier provided by scikit learn.<jupyter_code>import math
from scipy.io import wavfile
from scipy.fft import fft
import matplotlib.pyplot as plt
import numpy as np
import os
import pandas as pd
import seaborn as sn
from tqdm import tqdm


plt.rcParams['figure.facecolor'] = 'white'  # enhances visibility in darkmode IDEs
plt.rcParams["figure.figsize"] = (1.96, 1.96)
plt.rcParams['font.size'] = 4.2<jupyter_output><empty_output><jupyter_text>Data Import ***Walk over a directory determined by a given path and append the datapaths of all found files to a list.<jupyter_code>_DATAPATHS = []
rootdir = './bachelor_project/Data/Study Recordings'

for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        path = os.path.join(subdir, file)
        _DATAPATHS.append(path)

def fourier_transform(data):
    """
    Performs a fast fourier transformation for a given array of wav data.

    :param data: Array of frequency values gained from a wav recording.
    :return: The absolute values of the fourier transformed data.
    """
    transformed_data = fft(data)
    return np.abs(transformed_data[:int((len(transformed_data)/2)-1)])<jupyter_output><empty_output><jupyter_text>Generation of a Data Set---The following code block uses the previous imported data path to generate a data set for the latter analysis.For this the following steps are performed:   * Read sample rate and freqency data from a wave audio file   * Reshape the frequencies by applying a StandardScaler from scikit learn   * Perform the Fast Fourier Transformation for both original data and the scaled dataThe final data set contains the information gained from the steps described above as well as other key information about each recording.<jupyter_code>from sklearn.preprocessing import StandardScaler

sScaler = StandardScaler()
_DATASET = []

for d in _DATAPATHS:

    # Import the wave audio
    slr, wav_data = wavfile.read(d)

    # Apply StandardScaler to the audio data
    wav_data_standardized = sScaler.fit_transform(wav_data.reshape(-1,1))

    # Apply fft to both Standardized and normal audio data
    # Saved data gets cropped by end and beginning noise of the samples
    fft_data_nonStandard = fourier_transform(np.asarray(wav_data))[5:200000]
    fft_data_Standardized = fourier_transform(np.asarray(wav_data_standardized))[5:200000]

    # Append key informations and audio data to the data set
    _DATASET.append({'path': d,
                    'pid': int(os.path.basename(d).split('_')[0].replace('p', '')),
                    'session': int(os.path.basename(d).split('_')[1].replace('s', '')),
                    'hand': os.path.basename(d).split('_')[2].replace('u', ''),
                    'melody_type': os.path.basename(d).split('_')[3].replace('.wav', '')[0:-1],
                    'samplerate': slr,
                    'wav': wav_data,
                    'wav_standardized': wav_data_standardized,
                    'fft_nonStandard': fft_data_nonStandard,
                    'fft_wthStandard': fft_data_Standardized})<jupyter_output><empty_output><jupyter_text>Plot graphics for the audio data---The following three code blocks are each used to plot different variations of the audios from the data set.   1. Plots of the fft data for all entries of the data set   2. Plots of the wav audio for all entries of the data set   3. Plots of all variations of a specific sample grouped by their ID<jupyter_code>for d in tqdm(_DATASET):

    plt.title(d['path'].split('_u')[-1].replace('.wav', '_fft'))
    plt.plot(d['fft_wthStandard'], label='p1')
    plt.ticklabel_format(useOffset=False, style='plain')
    plt.savefig(d['path'].split('\\')[-1].replace('.wav', '_fftNorm.pdf'))
    plt.clf()

for d in tqdm(_DATASET):

    plt.title(d['path'].split('_u')[-1].replace('.wav', ''))

    data = d['wav']
    length = data.shape[0] / d['samplerate']
    time = np.linspace(0, length, data.shape[0])

    plt.plot(time, data)
    plt.savefig(d['path'].split('\\')[-1].replace('.wav', 'raw_plot.pdf'))
    plt.clf()

plt.title('left_high_noNorm')
plt.plot(_DATASET[0]['fft_nonStandard'], label='p0h1')
plt.plot(_DATASET[1]['fft_nonStandard'], label='p0h2')
plt.plot(_DATASET[2]['fft_nonStandard'], label='p0h3')
plt.ticklabel_format(useOffset=False, style='plain')
plt.legend()
plt.savefig('p0_allHighs_left_noNorm.pdf')<jupyter_output><empty_output><jupyter_text>Classification of the Audio data---Runs a classifier over all entries of the data set.For the classifier an ExtraTreesClassifier with a number of 1000 estimators is used.The audio data is sorted by tho keys (hand and sample type/melody).Training and validation set are split by the session ID of each entry.As final step the confusion matrices for each classification report are printed and stored as a pdf file.<jupyter_code>from sklearn.ensemble import ExtraTreesClassifier
from sklearn.metrics import classification_report, confusion_matrix, accuracy_score


for melody in ['high', 'low', 'mel']:
    for hand in ['r', 'l']:

        x_train = np.array([d['fft_nonStandard'] for d in _DATASET if d['session'] == 0 and d['melody_type'] == melody and d['hand'] == hand]).reshape(9,-1)
        x_val = np.array([d['fft_nonStandard'] for d in _DATASET if d['session'] == 1 and d['melody_type'] == melody and d['hand'] == hand]).reshape(9,-1)

        y_train = np.array([d['pid'] for d in _DATASET if d['session'] == 0 and d['melody_type'] == melody and d['hand'] == hand])
        y_val = np.array([d['pid'] for d in _DATASET if d['session'] == 1 and d['melody_type'] == melody and d['hand'] == hand])


        clf = ExtraTreesClassifier(n_estimators=1000, random_state=0)
        clf.fit(x_train, y_train)

        y_pred = clf.predict(x_val)

        report = classification_report(y_val, y_pred)
        cm = confusion_matrix(y_val, y_pred)
        
        print(melody, hand)
        accuracy = accuracy_score(y_train, y_pred, normalize=True)*100
        print(f'Accurancy: {accuracy: .4f}%')
        print(report)
        print(cm)
        labels = ['p_0', 'p_1', 'p_2']

        df_cm = pd.DataFrame(cm, index = labels, columns = labels)
        plt.figure(figsize= (3,3))
        sn_plot = sn.heatmap(df_cm, annot=True, cmap='Blues')
        plt.xlabel("True Label")
        plt.ylabel("Predicted Label")

        plt.savefig(f'confusion_matrix_{melody}_{hand}.pdf')
        plt.show()<jupyter_output>high r
Accurancy:  33.3333%
              precision    recall  f1-score   support

           0       0.29      0.67      0.40         3
           1       0.00      0.00      0.00         3
           2       0.50      0.33      0.40         3

    accuracy                           0.33         9
   macro avg       0.26      0.33      0.27         9
weighted avg       0.26      0.33      0.27         9

[[2 0 1]
 [3 0 0]
 [2 0 1]]
{'fraction_non_alphanumeric': 0.11789851075565361, 'fraction_numerical': 0.020959735245449532, 'mean_word_length': 3.335325762104005, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '555186', 'license': 'MIT', 'n_tokens_mistral': 2411, 'n_tokens_neox': 2305, 'n_words': 680, 'path': 'Study/Data Analysis/data_analysis.ipynb', 'repo_name': 'JohannesWaltmann/bachelor_project', 'size': '7252'}