<jupyter_start><jupyter_text>Read Midi File and Transform Chords to PC-Sets with music21 Reading a Midi file and finding chords in music21 setting<jupyter_code>from music21 import *
import ast<jupyter_output><empty_output><jupyter_text>Read Midi File<jupyter_code>mfile = converter.parse('chopin_raindrop_prelude.mid')<jupyter_output><empty_output><jupyter_text>Take all tracks and render them into vertical Chords<jupyter_code>mChords = mfile.chordify()<jupyter_output><empty_output><jupyter_text>Transform chords into PC-Sets __Notes for this step__  : We want to filter vertically all the chords such that if there are only two notes they will not be taken into account or if a chord doesn't produce a connected graph then we have to force one. Or if we have a note delay we have to force the correct one.We also take a list of the PrimeForm of chords. __TODO :__ Filter duplicates before taking the Primelist<jupyter_code>chordList = []
chordPrime = []
for c in mChords.recurse().getElementsByClass('Chord'):
    chordList.append(c.orderedPitchClasses)
    chordPrime.append(c.primeForm)<jupyter_output><empty_output><jupyter_text>Store PC-Sets in a list and count the elements<jupyter_code>print(chordList)
howManyChords = len(chordList)
print("There is total of", howManyChords, "Chords.")<jupyter_output>[[1, 5, 8], [1, 5, 8], [1, 5, 8], [1, 5, 8], [1, 5, 8], [1, 5, 8], [1, 5, 8], [1, 5, 6, 8, 10], [1, 5, 6, 8, 10], [1, 5, 6, 8, 10], [1, 6, 8, 10], [1, 5, 6, 10], [1, 5, 6, 8, 10], [1, 6, 8, 10], [1, 6, 8, 10], [0, 3, 6, 8], [0, 3, 6, 8], [0, 3, 6, 8], [0, 3, 6, 8], [0, 3, 6, 8], [0, 1, 3, 6, 8, 10], [0, 1, 3, 6, 10], [0, 1, 3, 6, 8, 10], [0, 1, 3, 5, 6, 8, 10], [1, 5, 6, 8, 10], [1, 5, 6, 10], [1, 6, 8, 10], [1, 6, 8, 10], [0, 3, 6], [0, 3, 6, 8], [0, 3, 5, 6, 8], [0, 3, 5, 6, 8, 10], [0, 3, 6, 8, 10], [0, 3, 6, 8, 10], [0, 1, 5, 6, 8, 10], [0, 1, 5, 6, 8, 10], [0, 1, 5, 6, 8, 10], [0, 1, 5, 8], [1, 5, 8], [1, 5, 8], [1, 5, 8], [1, 5, 8], [1, 5, 8], [0, 5, 8], [0, 5, 8], [0, 5, 6, 8], [0, 5, 6, 8], [0, 5, 6, 8], [0, 3, 5, 8], [1, 3, 5, 8], [1, 5, 8], [1, 5, 8], [1, 5, 8], [0, 1, 3, 5, 6, 8], [0, 1, 5, 6, 8], [0, 1, 5, 6, 8], [0, 1, 5, 6, 8], [0, 1, 3, 5, 6, 8], [0, 1, 5, 6, 8], [1, 2, 5, 8], [1, 3, 5, 8], [1, 5, 8], [1, 5, 6, 8], [1, 5, 8], [1, 5, 8], [1, 5, 8], [1, 5, 8], [1, 5, 8], [[...]<jupyter_text>Some Plots for good measure! (you can skip that if you don't want to lose time)<jupyter_code>graph.plot.HorizontalBarPitchClassOffset(mChords).run()
graph.plot.HistogramPitchClass(mChords).run()<jupyter_output><empty_output><jupyter_text>Filter the Chords : If two consecutive chords are the same then abstruct the last<jupyter_code>def removeDoubles(l):
    N = len(l)
    nl = []
    sl = [str(i) for i in l]
    nl.append(sl[0])
    for i in range(1,N):
        if sl[i] != sl[i-1]:
           nl.append(sl[i]) 
    return nl
x = removeDoubles(chordList)<jupyter_output><empty_output><jupyter_text>Find most Frequent chord<jupyter_code>def most_frequent(List): 
    counter = 0
    num = List[0] 
      
    for i in List: 
        curr_frequency = List.count(i) 
        if(curr_frequency> counter): 
            counter = curr_frequency 
            num = i 
  
    return num 
most_frequent(x)<jupyter_output><empty_output><jupyter_text>Transform the filtered pc-sets to chord C E G form<jupyter_code>def pcsetToNotes(l):
    ml = []
    nl = []

    for i in l:
        nl.append(ast.literal_eval(i))
    for i in nl:
        ml.append(chord.Chord(i))
    return ml
bn = pcsetToNotes(x)<jupyter_output><empty_output><jupyter_text>Interval Vectors :<jupyter_code>def interVec(l):
    lp = []
    for c in l:
        lp.append(c.intervalVector)
    return lp
interVec(bn)<jupyter_output><empty_output><jupyter_text>Stats on Covariance between consecutive chords<jupyter_code>import numpy as np
def getcov1(l):
	#convert list of lists to list of strings to make it hashable
    sl=[str(i) for i in l]
    #remove duplicates
    setl = set(sl)

    N = len(l) #length of piece
    dl = { j:i for i,j in enumerate(setl)} #dictionary from cords to index
    # list of cords based on their covariance index 
    invdl = len(setl)*[None] 
    for k in dl.keys():
        invdl[dl[k]]=k

    # get covariance beetween consecutive chords
    numcords = len(setl)
    cv = np.zeros((numcords,numcords))
    for i in range(1,N):
        cv[dl[sl[i-1]],dl[sl[i]]]+=1
    cv/=N
    return cv,dl,invdl
def getcov2(l):
	#convert list of lists to list of strings to make it hashable
    sl=[str(i) for i in l]
    #remove duplicates
    setl = set(sl)

    N = len(l) #length of piece
    dl = { j:i for i,j in enumerate(setl)} #dictionary from cords to index
    # list of cords based on their covariance index 
    invdl = len(setl)*[None] 
    for k in dl.keys():
        invdl[dl[k]]=k

    # get covariance between n and n-2
    numcords = len(setl)
    cv = np.zeros((numcords,numcords))
    for i in range(2,N):
        cv[dl[sl[i-2]],dl[sl[i]]]+=1
    cv/=N
    return cv,dl,invdl<jupyter_output><empty_output><jupyter_text>Get Principle Component Analysis<jupyter_code>def pca(x):
    cv,dl ,invdl = getcov1(x)
    vals,vecs = np.linalg.eigh(cv)
    inds = np.argsort(vals)[::-1]
    vecs = vecs[inds]
    vals = vals[inds]
    return vals,vecs,dl,invdl<jupyter_output><empty_output><jupyter_text>Some Graphs<jupyter_code>import matplotlib.pyplot as plt
cv, dl, invdl= getcov1(x)
plt.imshow(cv)
plt.set_cmap('hot')
plt.colorbar()<jupyter_output><empty_output><jupyter_text>Get the max elements for the Covarience matrix in all the lists<jupyter_code>ind = np.unravel_index(np.argmax(cv, axis=None), cv.shape)
invdl[ind[0]], invdl[ind[1]]
result = np.where(cv == np.amax(cv))
print(result)
def getKeysByValue(dictOfElements, valueToFind):
    listOfKeys = list()
    listOfItems = dictOfElements.items()
    for item  in listOfItems:
        if item[1] == valueToFind:
            listOfKeys.append(item[0])
    return  listOfKeys
getKeysByValue(dl, ind[0]), getKeysByValue(dl, ind[1])<jupyter_output><empty_output><jupyter_text>An attemp to find the key of the piece based on the Krumhansl-Schmuckler key-finding algorithm. Statistics about the Key, discrete analysis tools and duration of cords<jupyter_code>pitchHistogram = features.jSymbolic.PitchClassDistributionFeature(mChords)
f = pitchHistogram.extract()
f.vector<jupyter_output><empty_output><jupyter_text>Discete analysis tools for Key_finding<jupyter_code>analysis.discrete.analyzeStream(mChords, 'Krumhansl')
analysis.discrete.analyzeStream(mChords, 'key')<jupyter_output><empty_output><jupyter_text>This takes a lot of time so be patient :<jupyter_code>def durationValues(x):
    avDur = features.jSymbolic.AverageNoteDurationFeature(x)
    f1 = avDur.extract()
    minDur = features.jSymbolic.MinimumNoteDurationFeature(x)
    f2 = minDur.extract()
    maxDur = features.jSymbolic.MaximumNoteDurationFeature(x)
    f3 = maxDur.extract()
    return f1.vector, f2.vector, f3.vector
durationValues(mChords)<jupyter_output><empty_output><jupyter_text>Discete Overlapped window analysis Break down my data into smaller windows for analysis<jupyter_code>def windowedAnalysis(x):
    p = analysis.discrete.Ambitus()
    wa = analysis.windowed.WindowedAnalysis(x, p)
    return wa.analyze(4)<jupyter_output><empty_output><jupyter_text>Takes toooooo looong something's not right<jupyter_code>windowedAnalysis(mChords)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14802852913470596, 'fraction_numerical': 0.04548512986139147, 'mean_word_length': 4.193570929419986, 'pattern_counts': {'":': 0, '<': 76, '<?xml version=': 0, '>': 77, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '232243', 'license': 'MIT', 'n_tokens_mistral': 3223, 'n_tokens_neox': 2898, 'n_words': 835, 'path': '00_Read_Midi_Files.ipynb', 'repo_name': 'melkisedeath/Harmonic_Analysis_and_Trajectory', 'size': '7431'}