<jupyter_start><jupyter_text>Table of Contents* [Exploratory Data Analysis](Header)    - [Metadata](Metadata)        - [Missing Values](MissingValues)        - [Species](Species)        - [Date and Time](DateTime)        - [Recordists](Recordists)        - [Location](Location)* [Audio Feature Extraction](AudioFeatureExtraction)    - [Waveform](Waveform)    - [Autocorrelation](Autocorrelation)    - [Spectrogram](Spectrogram)    - [Chromagram](Chromagram)    - [Spectral](Spectral)        - [Centroid](Centroid)        - [Bandwidth](Bandwidth)        - [Contrast](Contrast)        - [Flatness](Flatness)        - [Rolloff](Rolloff)    - [MFCC](MFCC)* [Afterword](Thanks)  Bird Call Classification: Data Exploration   Environment<jupyter_code>import os
import sys
import librosa
import librosa.display
import librosa.feature
import numpy as np
import pandas as pd
import plotly.express as xp
import plotly.graph_objects as go
import matplotlib.pyplot as plt
import IPython.display as ipd
from sklearn.preprocessing import minmax_scale

import warnings
warnings.filterwarnings('ignore')

%matplotlib inline<jupyter_output><empty_output><jupyter_text>MetadataRelevant Features- ebird_code- date/time- location- recordist- filename<jupyter_code>train = pd.read_csv('../input/birdsong-recognition/train.csv')
train.head()
train.info()<jupyter_output><empty_output><jupyter_text>Missing ValuesAnalyze missing data and determine cleaning steps.<jupyter_code>missing = train.isna().sum().sort_values(ascending=False)
missing = missing[missing != 0]
xp.bar(x=missing.index, y=missing, text=missing, title='Missing Values by Feature', labels={'x':'Features', 'y':'Quantity'})<jupyter_output><empty_output><jupyter_text>- No relevant data missing  SpeciesDistribution of bird species<jupyter_code>counts = train['ebird_code'].value_counts()
xp.bar(x=counts.index, y=counts, title='Species Distribution (by ebird code)', labels={'x':'Ebird Code', 'y':'Quantity'})<jupyter_output><empty_output><jupyter_text>- Exactly 100 samples for about half of species in question- Redhead is minimum at 9 samples  Date and TimeRecording Date Distribution<jupyter_code># split datetime into separate dataframe
datetime = train[['date', 'time']]
datetime.date = pd.to_datetime(datetime.date, errors='coerce').dropna()
datetime['hour'] = pd.to_numeric(datetime.time.str.split(':', expand=True)[0], errors='coerce')
ax1 = datetime.date.value_counts().sort_values().plot(figsize=(10,6), title='Recordings by Date')

ax1.set_xlabel('Date')
ax1.set_ylabel('Quantity')
plt.show()<jupyter_output><empty_output><jupyter_text>- Majority of recordings taken in the past decade- Interesting spike around 2003- Cyclical spikes after 2013<jupyter_code>ax2 = datetime['hour'].value_counts().sort_index().plot(figsize=(10,6), title='Recordings by Time', kind='bar', figure=plt.figure())

ax2.set_xlabel('Hour')
ax2.set_ylabel('Quantity')
plt.show()<jupyter_output><empty_output><jupyter_text>- Most recordings taken between 6AM and 12PM- Gradual decrease as the day moves on from 8AM  RecordistsWho recorded the samples?<jupyter_code>ax3 = train['recordist'].value_counts().sort_values(ascending=False).head(20).sort_values().plot(figsize=(10, 6), title='Recordings by Recordist', figure=plt.figure(), kind='barh', fontsize=9)

ax3.set_xlabel('Hour')
ax3.set_ylabel('Quantity')
plt.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>- Majority of recordings made by only two people  Location<jupyter_code>counts = train['country'].value_counts().sort_values(ascending=False).head(10).sort_values()
xp.bar(y=counts.index, x=counts, title='Number of Recordings by Country', labels={'y':'Country', 'x':'Quantity'}, orientation='h')
coords = train.groupby(['latitude', 'longitude'], as_index=False)['ebird_code'].agg('count')
coords = coords[coords.latitude != 'Not specified']
coords = coords[coords.longitude != 'Not specified']
xp.scatter_geo(lat=coords['latitude'], lon=coords['longitude'], title='Recording Locations', size=coords['ebird_code'])<jupyter_output><empty_output><jupyter_text>- Vast majority of data comes from North America, specifically from USA  Audio Feature Extraction SampleFirst, let's take the first 5 audio samples from train.csv and explore.<jupyter_code>bird_codes = train.ebird_code.unique()[:5]

audio = []
for bird in range(len(bird_codes)):
    filename = train[train['ebird_code'] == bird_codes[bird]]['filename'].iloc[0]
    path = os.path.join('../input/birdsong-recognition/train_audio/', bird_codes[bird], filename)
    
    # wave plot
    plt.figure(figsize=(15,10))
    plt.subplot(len(bird_codes), 1, bird+1)
    data, srate = librosa.load(path)
    librosa.display.waveplot(data, sr=srate)
    plt.gca().set_title(bird_codes[bird])
    plt.xticks([],[])
    plt.xlabel('')
    plt.show()
    
    # audio display
    audio = ipd.Audio(path)
    ipd.display(audio)<jupyter_output><empty_output><jupyter_text>FeaturesAfter doing some research on audio signal classification, I have come up with the following features to extract from the audio files:- [Waveform](Waveform)- [Autocorrelation](Autocorrelation)- [Spectrogram](Spectrogram)- [Chromagram](Chromagram)- [Spectral](Spectral)    - [Centroid](Centroid)    - [Bandwidth](Bandwidth)    - [Contrast](Contrast)    - [Flatness](Flatness)    - [Rolloff](Rolloff)- [MFCC](MFCC)We'll do a sample feature extraction of bird code 'ameavo' as an example. Waveform<jupyter_code>data, srate = librosa.load('../input/birdsong-recognition/train_audio/ameavo/XC99571.mp3')
# plot waveform as refresher
plt.figure(figsize=(15,5))
librosa.display.waveplot(data, sr=srate)
plt.gca().set_title('ameavo')
plt.xticks([],[])
plt.xlabel('')
plt.show()<jupyter_output><empty_output><jupyter_text>AutocorrelationAutocorrelation compares a signal with a lagged version of itself. It is often used as a measure of periodicity in a signal.<jupyter_code>autocorrelation = librosa.autocorrelate(data, max_size=5000)
plt.figure(figsize=(15,5))
plt.plot(autocorrelation)
plt.gca().set_title('Autocorrelation by Lag Time')
plt.xlabel('Lag')
plt.show()<jupyter_output><empty_output><jupyter_text>- autocorrelation very quickly falls off reaching almost 0 after a lag of about 500  SpectrogramThe spectrogram is a visual representation of a signal's spectrum of frequencies over time.<jupyter_code>spectrogram = librosa.stft(data)
plt.figure(figsize=(20,10))
librosa.display.specshow(librosa.amplitude_to_db(abs(spectrogram)), sr=srate, x_axis='time', y_axis='hz')
plt.xlabel('Time', fontsize=20)
plt.ylabel('Frequency Band')
plt.colorbar()
plt.title('Spectrogram', fontsize=20)
plt.show()<jupyter_output><empty_output><jupyter_text>- Pitch seems to hover around 2000 to 3500 Hz most of the time- Some spikes to 5500-7000 Hz  ChromagramThe Chromagram is a visual representation of a signal's chroma feature. The chroma feature at any point in time is the intensity for each chroma value in the set {C, C♯, D, D♯, E , F, F♯, G, G♯, A, A♯, B}. These values are the various rows of the chromagram.<jupyter_code>chroma = librosa.feature.chroma_stft(data, sr=srate)
plt.figure(figsize=(20,10))
librosa.display.specshow(chroma, x_axis='time', y_axis='chroma')
plt.xlabel('Time', fontsize=20)
plt.ylabel('Chroma Value')
plt.colorbar()
plt.clim(0,1)
plt.title('Chromagram', fontsize=20)
plt.show()<jupyter_output><empty_output><jupyter_text>Spectral Features Spectral CentroidSpectral Centroid is a measurement of the "center of gravity" of the signal and is a common metric of timbre in a sound sample. It's essentially the dominant frequency at each point.<jupyter_code>centroid = librosa.feature.spectral_centroid(data)[0]
plt.figure(figsize=(15,5))
librosa.display.waveplot(data, sr=srate)
plt.plot(librosa.frames_to_time(range(len(centroid))), minmax_scale(centroid), color='g')
plt.gca().set_title('Spectral Centroid by Frame')
plt.xlabel('Frame')
plt.show()<jupyter_output><empty_output><jupyter_text>Spectral BandwidthSpectral bandwidth represents the range between the lowest and highest values of the signal at a certain time.<jupyter_code>bandwidth = librosa.feature.spectral_bandwidth(data, sr=srate)[0]
plt.figure(figsize=(15,5))
librosa.display.waveplot(data, sr=srate)
plt.plot(librosa.frames_to_time(range(len(bandwidth))), minmax_scale(bandwidth))
plt.gca().set_title('Spectral Bandwidth by Time')
plt.xlabel('Time')
plt.show()<jupyter_output><empty_output><jupyter_text>- pure noise portions of sample are higher in bandwidth Spectral ContrastSpectral contrast compares the max and min frequency values for each frequency band at a point in time. Thus, spectral contrast gives a robust measure of relative spectral characteristics.<jupyter_code>contrast = librosa.feature.spectral_contrast(data, sr=srate)
plt.figure(figsize=(20,10))
librosa.display.specshow(contrast, x_axis='time')
plt.xlabel('Time', fontsize=20)
plt.colorbar()
plt.title('Spectral Contrast', fontsize=20)
plt.ylabel('Frequency Band', fontsize=20)
plt.show()<jupyter_output><empty_output><jupyter_text>- highest contrast occurs in the lowest and highest frequency bands  Spectral FlatnessSpectral flatness compares the arithmetic and geometric means of the power spectrum. It is most often used to identify and separate tones versus noise.<jupyter_code>flatness = librosa.feature.spectral_flatness(data)
plt.figure(figsize=(20,10))
librosa.display.specshow(flatness, x_axis='time')
plt.xlabel('Time', fontsize=20)
plt.colorbar()
plt.clim(0,1)
plt.title('Spectral Flatness', fontsize=20)
plt.ylabel('Frequency Band', fontsize=20)
plt.show()<jupyter_output><empty_output><jupyter_text>- Maximum 20% noise at points in sample.- Very low noise in general Spectral RolloffSpectral rolloff is the frequency under which a specified percentage of the energy lies<jupyter_code>rolloff = librosa.feature.spectral_rolloff(data, sr=srate)[0]
plt.figure(figsize=(15,5))
librosa.display.waveplot(data, sr=srate)
plt.plot(librosa.frames_to_time(range(len(rolloff))), minmax_scale(rolloff))
plt.gca().set_title('Spectral Bandwidth by Time')
plt.xlabel('Time')
plt.show()<jupyter_output><empty_output><jupyter_text>MFCCMel-Frequency Cepstral Coefficients are a collection of coefficients that together give a representation of the overall spectral envelope of a signal. Probably the most common and important feature of audio signal processing in machine learning.<jupyter_code>mfcc = librosa.feature.mfcc(data, sr=srate, n_mfcc=30)
plt.figure(figsize=(20,10))
librosa.display.specshow(minmax_scale(mfcc, axis=1), x_axis='time')
plt.xlabel('Time', fontsize=20)
plt.colorbar()
plt.clim(0,1)
plt.title('Mel-Frequency Cepstral Coefficients', fontsize=20)
plt.show()

print()
print('MFCCs calculated: %d' % mfcc.shape[0])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13195607667969478, 'fraction_numerical': 0.014330913828401265, 'mean_word_length': 6.996279761904762, 'pattern_counts': {'":': 0, '<': 77, '<?xml version=': 0, '>': 77, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '145311', 'license': 'MIT', 'n_tokens_mistral': 3804, 'n_tokens_neox': 3530, 'n_words': 814, 'path': 'Notebooks/eda.ipynb', 'repo_name': 'CSLukeW/Cornell-Birdcall-Identification', 'size': '10746'}