<jupyter_start><jupyter_text>ThinkDSPThis notebook contains code examples from Chapter 5: AutocorrelationCopyright 2015 <NAME>License: [Creative Commons Attribution 4.0 International](http://creativecommons.org/licenses/by/4.0/)<jupyter_code>from __future__ import print_function, division

import thinkdsp
import thinkplot
import thinkstats2 

import numpy as np

import warnings
warnings.filterwarnings('ignore')

from ipywidgets import interact, interactive, fixed
import ipywidgets as widgets

PI2 = np.pi * 2

%matplotlib inline<jupyter_output><empty_output><jupyter_text>To investigate serial correlation of signals, let's start with a sine wave at 440 Hz.<jupyter_code>def make_sine(offset):
    signal = thinkdsp.SinSignal(freq=440, offset=offset)
    wave = signal.make_wave(duration=0.5, framerate=10000)
    return wave<jupyter_output><empty_output><jupyter_text>I'll make two waves with different phase offsets.<jupyter_code>wave1 = make_sine(offset=0)
wave2 = make_sine(offset=1)

thinkplot.preplot(2)
wave1.segment(duration=0.01).plot()
wave2.segment(duration=0.01).plot()
thinkplot.config(xlabel='Time (s)', ylim=[-1.05, 1.05])<jupyter_output><empty_output><jupyter_text>The two waves appears correlated: when one is high, the other is usually high, too.We can use `np.corrcoef` to compute the correlation matrix.<jupyter_code>print(np.corrcoef(wave1.ys, wave2.ys))<jupyter_output>[[ 1.          0.54030231]
 [ 0.54030231  1.        ]]<jupyter_text>The diagonal elements are the correlations of the waves with themselves, which is why they are 1.The off-diagonal elements are the correlations between the two waves.  In this case, 0.54 indicates that there is a moderate correlation between these waves.The correlation matrix is more interesting when there are more than two waves.  With only two waves, there is really only one number in the matrix we care about.`thinkdsp.Wave` provides `corr`, which computes the correlation between waves:<jupyter_code>wave1.corr(wave2)<jupyter_output><empty_output><jupyter_text>To investigate the relationship between phase offset and correlation, I'll make an interactive function that computes correlation for each offset:<jupyter_code>def compute_corr(offset):
    wave1 = make_sine(offset=0)
    wave2 = make_sine(offset=-offset)
    
    thinkplot.preplot(2)
    wave1.segment(duration=0.01).plot()
    wave2.segment(duration=0.01).plot()
    
    corr = wave1.corr(wave2)
    print('corr =', corr)
    
    thinkplot.config(xlabel='Time (s)', ylim=[-1.05, 1.05])<jupyter_output><empty_output><jupyter_text>The following interaction plots waves with different phase offsets and prints their correlations:<jupyter_code>slider = widgets.FloatSlider(min=0, max=PI2, value=1)
interact(compute_corr, offset=slider);<jupyter_output>corr = 0.540302305868<jupyter_text>Finally, we can plot correlation as a function of offset:<jupyter_code>offsets = np.linspace(0, PI2, 101)

corrs = []
for offset in offsets:
    wave2 = make_sine(offset)
    corr = np.corrcoef(wave1.ys, wave2.ys)[0, 1]
    corrs.append(corr)
    
thinkplot.plot(offsets, corrs)
thinkplot.config(xlabel='Offset (radians)',
                 ylabel='Correlation', 
                 axis=[0, PI2, -1.05, 1.05])<jupyter_output><empty_output><jupyter_text>That curve is a cosine.Next we'll compute serial correlations for different kinds of noise.<jupyter_code>def serial_corr(wave, lag=1):
    N = len(wave)
    y1 = wave.ys[lag:]
    y2 = wave.ys[:N-lag]
    corr = np.corrcoef(y1, y2, ddof=0)[0, 1]
    return corr<jupyter_output><empty_output><jupyter_text>We expect uncorrelated noise to be... well... uncorrelated.<jupyter_code>signal = thinkdsp.UncorrelatedGaussianNoise()
wave = signal.make_wave(duration=0.5, framerate=11025)
serial_corr(wave)<jupyter_output><empty_output><jupyter_text>As expected, the serial correlation is small.In Brownian noise, each value is the sum of the previous value and a random "step", so we expect a strong serial correlation:<jupyter_code>signal = thinkdsp.BrownianNoise()
wave = signal.make_wave(duration=0.5, framerate=11025)
serial_corr(wave)<jupyter_output><empty_output><jupyter_text>In fact, the correlation is near 1.Since pink noise is between white and Brownian, we expect an intermediate correlation.<jupyter_code>signal = thinkdsp.PinkNoise(beta=1)
wave = signal.make_wave(duration=0.5, framerate=11025)
serial_corr(wave)<jupyter_output><empty_output><jupyter_text>And we get one.Now we can plot serial correlation as a function of the pink noise parameter $\beta$.<jupyter_code>np.random.seed(19)

betas = np.linspace(0, 2, 21)
corrs = []

for beta in betas:
    signal = thinkdsp.PinkNoise(beta=beta)
    wave = signal.make_wave(duration=1.0, framerate=11025)
    corr = serial_corr(wave)
    corrs.append(corr)
    
thinkplot.preplot(1)
thinkplot.plot(betas, corrs)
thinkplot.config(xlabel=r'Pink noise parameter, $\beta$',
                 ylabel='Serial correlation', 
                 ylim=[0, 1.05])<jupyter_output><empty_output><jupyter_text>The autocorrelation function calls `serial_corr` with different values of `lag`.<jupyter_code>def autocorr(wave):
    """Computes and plots the autocorrelation function.

    wave: Wave
    """
    lags = range(len(wave.ys)//2)
    corrs = [serial_corr(wave, lag) for lag in lags]
    return lags, corrs<jupyter_output><empty_output><jupyter_text>Now we can plot autocorrelation for pink noise with various values of $\beta$.<jupyter_code>def plot_pink_autocorr(beta, label):
    signal = thinkdsp.PinkNoise(beta=beta)
    wave = signal.make_wave(duration=1.0, framerate=10000)
    lags, corrs = autocorr(wave)
    thinkplot.plot(lags, corrs, label=label)
np.random.seed(19)
thinkplot.preplot(3)

for beta in [1.7, 1.0, 0.3]:
    label = r'$\beta$ = %.1f' % beta
    plot_pink_autocorr(beta, label)

thinkplot.config(xlabel='Lag', 
                 ylabel='Correlation',
                 xlim=[-1, 1000], 
                 ylim=[-0.05, 1.05],
                 legend=True)<jupyter_output><empty_output><jupyter_text>For low values of $\beta$, the autocorrelation function drops off quickly.  As $\beta$ increases, pink noise shows more long range dependency. Now let's investigate using autocorrelation for pitch tracking.  I'll load a recording of someone singing a chirp:<jupyter_code>wave = thinkdsp.read_wave('28042__bcjordan__voicedownbew.wav')
wave.normalize()
wave.make_audio()<jupyter_output><empty_output><jupyter_text>The spectrum tells us what frequencies are present, but for chirps, the frequency components are blurred over a range:<jupyter_code>spectrum = wave.make_spectrum()
spectrum.plot()
thinkplot.config(xlabel='Frequency (Hz)', ylabel='Amplitude')<jupyter_output><empty_output><jupyter_text>The spectrogram gives a better picture of how the components vary over time:<jupyter_code>spectro = wave.make_spectrogram(seg_length=1024)
spectro.plot(high=4200)
thinkplot.config(xlabel='Time (s)', 
                 ylabel='Frequency (Hz)',
                 xlim=[wave.start, wave.end])<jupyter_output><empty_output><jupyter_text>We can see the fundamental frequency clearly, starting near 500 Hz and dropping.  Some of the harmonics are also visible.To track the fundamental frequency, we can take a short window:<jupyter_code>duration = 0.01
segment = wave.segment(start=0.2, duration=duration)
segment.plot()
thinkplot.config(xlabel='Time (s)', ylim=[-1, 1])

spectrum = segment.make_spectrum()
spectrum.plot(high=1000)
thinkplot.config(xlabel='Frequency (Hz)', ylabel='Amplitude')<jupyter_output><empty_output><jupyter_text>The spectrum shows a clear peak near 400 Hz, but we can't get an very accurate estimate of frequency, partly because the peak is blurry, and partly because even if it were a perfect spike, the frequency resolution is not very good.<jupyter_code>len(segment), segment.framerate, spectrum.freq_res<jupyter_output><empty_output><jupyter_text>Each element of the spectrum spans a range of 100 Hz, so we can't get an accurate estimate of the fundamental frequency.  For signals that are at least approximately periodic, we can do better by estimating the length of the period.The following function plots the segment, and a shifted version of the segment, and computes the correlation between them:<jupyter_code>def plot_shifted(wave, offset=0.001, start=0.2):
    thinkplot.preplot(2)
    segment1 = wave.segment(start=start, duration=0.01)
    segment1.plot(linewidth=2, alpha=0.8)

    # start earlier and then shift times to line up
    segment2 = wave.segment(start=start-offset, duration=0.01)
    segment2.shift(offset)
    segment2.plot(linewidth=2, alpha=0.4)

    corr = segment1.corr(segment2)
    text = r'$\rho =$ %.2g' % corr
    thinkplot.text(segment1.start+0.0005, -0.8, text)
    thinkplot.config(xlabel='Time (s)', xlim=[start, start+duration], ylim=[-1, 1])

plot_shifted(wave, 0.0001)<jupyter_output><empty_output><jupyter_text>With a small shift the segments are still moderately correlated.  As the shift increases, the correlation falls for a while, then rises again, peaking when the shift equals the period of the signal.You can use the following interaction to search for the shift that maximizes correlation:<jupyter_code>end = 0.004
slider1 = widgets.FloatSlider(min=0, max=end, step=end/40, value=0)
slider2 = widgets.FloatSlider(min=0.1, max=0.5, step=0.05, value=0.2)
interact(plot_shifted, wave=fixed(wave), offset=slider1, start=slider2)
None<jupyter_output><empty_output><jupyter_text>The `autocorr` function automates this process by computing the correlation for each possible lag, up to half the length of the wave. The following figure shows this autocorrelation as a function of lag:<jupyter_code>wave = thinkdsp.read_wave('28042__bcjordan__voicedownbew.wav')
wave.normalize()
duration = 0.01
segment = wave.segment(start=0.2, duration=duration)
lags, corrs = autocorr(segment)
thinkplot.plot(lags, corrs)
thinkplot.config(xlabel='Lag (index)', ylabel='Correlation', ylim=[-1, 1])<jupyter_output><empty_output><jupyter_text>The first peak (other than 0) is near lag=100.We can use `argmax` to find the index of that peak:<jupyter_code>low, high = 90, 110
lag = np.array(corrs[low:high]).argmax() + low
lag<jupyter_output><empty_output><jupyter_text>We can convert from an index to a time in seconds:<jupyter_code>period = lag / segment.framerate
period<jupyter_output><empty_output><jupyter_text>Given the period in seconds, we can compute frequency:<jupyter_code>frequency = 1 / period
frequency<jupyter_output><empty_output><jupyter_text>This should be a better estimate of the fundamental frequency.  We can approximate the resolution of this estimate by computing how much we would be off by if the index were off by 1:<jupyter_code>segment.framerate / 102, segment.framerate / 100<jupyter_output><empty_output><jupyter_text>The range is less than 10 Hz.The function I wrote to compute autocorrelations is slow; `np.correlate` is much faster.<jupyter_code>N = len(segment)
corrs2 = np.correlate(segment.ys, segment.ys, mode='same')
lags = np.arange(-N//2, N//2)
thinkplot.plot(lags, corrs2)
thinkplot.config(xlabel='Lag', ylabel='Correlation', xlim=[-N//2, N//2])<jupyter_output><empty_output><jupyter_text>`np.correlate` computes correlations for positive and negative lags, so lag=0 is in the middle.  For our purposes, we only care about positive lags.Also, `np.correlate` doesn't correct for the fact that the number of overlapping elements changes as the lag increases.The following code selects the second half of the results and corrects for the length of the overlap:<jupyter_code>N = len(corrs2)
lengths = range(N, N//2, -1)

half = corrs2[N//2:].copy()
half /= lengths
half /= half[0]
thinkplot.plot(half)
thinkplot.config(xlabel='Lag', ylabel='Correlation', ylim=[-1.05, 1.05])<jupyter_output><empty_output><jupyter_text>Now the result is similar to what we computed before.If we plot the results computed by NumPy and my implementation, they are visually similar.  They are not quite identical because my version and theirs are normalized differently.<jupyter_code>thinkplot.preplot(2)
thinkplot.plot(half)
thinkplot.plot(corrs)
thinkplot.config(xlabel='Lag', ylabel='Correlation', ylim=[-1.05, 1.05])<jupyter_output><empty_output><jupyter_text>The difference between the NumPy implementation and mine is less than 0.02 over most of the range.<jupyter_code>diff = corrs - half[:-1]
thinkplot.plot(diff)
thinkplot.config(xlabel='Lag', ylabel='Difference in correlation')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1071228715324966, 'fraction_numerical': 0.028859221360620354, 'mean_word_length': 5.529227557411273, 'pattern_counts': {'":': 0, '<': 124, '<?xml version=': 0, '>': 124, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '31', 'dir': 'jupyter-structured-clean-dedup', 'id': '504187', 'license': 'MIT', 'n_tokens_mistral': 4331, 'n_tokens_neox': 4034, 'n_words': 1127, 'path': 'ThinkDSP-master/code/chap05.ipynb', 'repo_name': 'beckermy/popcorn-analyzer', 'size': '12515'}