<jupyter_start><jupyter_text>Holographic Reduced Representations This notebook has some basic tests and demonstrations of a pure Python 3 implementation of holographic reduced representations (HRRs) as outlined in [Plate 1995](http://ieeexplore.ieee.org/document/377968/). __The current implementation does not use numpy, so it's somewhat slow.__HRRs are a type of hetero-associative distributed representation that can efficiently encode nested compositional structure. Items are represented by n-tuples of floating-point elements, and encoded into pairs using circular convolution. The resulting n-tuples, called traces, may be composed together via element-wise addition to produce new traces. Importantly, HRR item "atoms" and HRR memory trace "molecules" are of the same dimension. This means traces may themselves function as items, allowing for the representation of the nested structure required by e.g. higher-order predicates. Other types of DRs represent memory traces and items in different spaces, or have exponentially increasing representation size with each composition. HRRs avoid this, at the cost of reduced representation fidelity and added limitations on the distribution from which element values are drawn. why?HRRs can be found in the recent development of differentiable neural computers, a geometric analogue of quantum computation, and distributed representations in cognitive science.---The notebook sections here are, roughly, demos of: - basic HRR encoding, decoding, composition - examples of representing frames and sentences from Plate 1995.and (probably of limited interest): - Sanity checks based on computed values compared to analytical results in Plate 1995  Table of Contents 1. [Basic encoding/decoding](basic)1. [Demos of more complex representations](Demos)    1. Sentences    1. Frames    1.1. [Some sanity checks](san)    1. element distribution    1. convolution working    1. basic encoding and decoding    1. computed xi and eta distributions -vs- analytical values1. Capacity of HRRs    1. computations    1. compared to other DRs1. Misc<jupyter_code>import hrr
#for simulations, plotting, etc.
import numpy as np
from matplotlib import pyplot as plt
import matplotlib.mlab as mlab
import math
from functools import reduce
from pandas import DataFrame

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Basic encoding, composition, and decoding<jupyter_code>#some dummy items to represent
items = ['h', 'i', 'j', 'k']

#associative memory matching each item to its HRR representation
M = {i:hrr.HRR(n_dims = 2048) for i in items}

#convolve two pairs of items, and compose the resulting traces
trace1 = M['j'].encode(M['k'])
trace2 = M['h'].encode(M['i'])
composed = trace1 + trace2

_i = trace2.decode(M['h'])
_h = trace2.decode(M['i'])
_k = trace1.decode(M['j'])
_j = trace1.decode(M['k'])

#decode each item's partner from the composed trace
i_ = composed.decode(M['h'])
h_ = composed.decode(M['i'])
k_ = composed.decode(M['j'])
j_ = composed.decode(M['k'])

print('Ranking of dot products of decoded traces with items in memory:')
print('_i:', hrr.getClosest(_i, M, 4))
print('_h:', hrr.getClosest(_h, M, 4))
print('_j:', hrr.getClosest(_j, M, 4))
print('_k:', hrr.getClosest(_k, M, 4))
print('i_:', hrr.getClosest(i_, M, 4))
print('h_:', hrr.getClosest(h_, M, 4))
print('j_:', hrr.getClosest(j_, M, 4))
print('k_:', hrr.getClosest(k_, M, 4))<jupyter_output>Ranking of dot products of decoded traces with items in memory:
_i: {'i': 1.00671, 'k': 0.02976, 'j': 0.01616, 'h': 0.01358}
_h: {'h': 1.00671, 'k': 0.04558, 'i': -0.01444, 'j': -0.022}
_j: {'j': 0.95465, 'i': -0.01791, 'h': -0.03365, 'k': -0.1172}
_k: {'k': 0.95465, 'h': 0.02525, 'i': -0.00298, 'j': -0.07946}
i_: {'i': 1.03659, 'j': 0.04141, 'h': 0.00145, 'k': -0.00389}
h_: {'h': 1.03659, 'k': 0.02767, 'i': -0.02004, 'j': -0.02498}
j_: {'j': 0.98452, 'i': 0.02767, 'h': -0.00389, 'k': -0.10906}
k_: {'k': 0.98452, 'h': 0.04141, 'i': -0.02498, 'j': -0.10008}<jupyter_text>Demos -- representing more complex structure  Representing sentences<jupyter_code>object_features = ['being', 'person', 'state', 'food', 'fish', 'bread']
role_features = ['object','agent']
frame_labels = ['cause', 'eat', 'see']
base_features = object_features + role_features + frame_labels

#some convenience lists -- not to be stored in memory 
_names = ['mark', 'john', 'paul', 'luke']
_foods = ['fish', 'bread']
_states = ['hunger', 'thirst']
identifiers = ['id_' + i for i in  _names + _foods + _states + [r + '_' + f for r in role_features for f in frame_labels]]
#base feature memory
M = {i:hrr.HRR() for i in identifiers+object_features+frame_labels+role_features}

# for k,v in M.items(): print(k, v)
# print(type((M['being'] + M['person'] + M['id_' + 'mark'])))

#memories for...
#people, e.g. mark
P = {p:(M['being'] + M['person'] + M['id_' + p])/math.sqrt(3) for p in _names} #/(math.sqrt(3))
#food, e.g. the_fish
F = {('the_' + f):(M['food'] + M[f] + M['id_' + f])/math.sqrt(3) for f in _foods}
#states, e.g. hunger
S = {s:(M['state'] + M['id_' + s])/math.sqrt(2) for s in _states}
#roles, e.g. agt_cause
R = {(r + '_' + f):(M[r]+M['id_' + r + '_' + f])/math.sqrt(2) for f in frame_labels for r in role_features}

#combine all into token and role memory
tokens = {**P, **F, **S, **R, **M}
T = tokens #alias to save some typing
               
#building sentences using the above tokens. Divide by sqrt(elements) to normalize vector lengths to 1
sentences = {
    1: (T['eat'] + T['agent_eat'].encode(T['mark']) + T['object_eat'].encode(T['the_fish']))/math.sqrt(3),
    3: (T['eat'] + T['agent_eat'].encode(T['john']))/math.sqrt(2),
    4: (T['see'] + T['agent_see'].encode(T['john']) + T['object_see'].encode(T['mark']))/math.sqrt(3),
    5: (T['see'] + T['agent_see'].encode(T['john']) + T['object_see'].encode(T['the_fish']))/math.sqrt(3),
    6: (T['see'] + T['agent_see'].encode(T['the_fish']) + T['object_see'].encode(T['john']))/math.sqrt(3)
}
sentences[2] = (T['cause'] + T['agent_cause'].encode(T['hunger']) + T['object_cause'].encode(sentences[1]))/math.sqrt(3)

T={**T, **sentences}
#similarities of tokens
token_order = _names + _foods + _states
token_sims = np.zeros((len(token_order), len(token_order)))
for tr in range(len(token_order)):
    for tc in range(tr+1):
        token_sims[tr][tc] = round(T[token_order[tr]] * T[token_order[tc]], 3)
        
#similarities of sentences
sentence_order = sorted(sentences.keys())
sentence_sims = np.zeros((len(sentences), len(sentences)))
for tr in range(len(sentence_order)):
    for tc in range(tr+1):
        sentence_sims[tr][tc] = round(sentences[sentence_order[tr]] * sentences[sentence_order[tc]], 3)


print("Token similarities (dot product):\n")
print(DataFrame(token_sims, token_order, token_order), '\n')
print("Sentence similarities (dot product):\n")
print(DataFrame(sentence_sims, ["S"+str(i) for i in sentence_order], ["S"+str(i) for i in sentence_order]))<jupyter_output>Token similarities (dot product):

         mark   john   paul   luke   fish  bread  hunger  thirst
mark    1.010  0.000  0.000  0.000  0.000  0.000   0.000   0.000
john    0.698  1.079  0.000  0.000  0.000  0.000   0.000   0.000
paul    0.694  0.754  1.022  0.000  0.000  0.000   0.000   0.000
luke    0.678  0.709  0.714  1.050  0.000  0.000   0.000   0.000
fish    0.012  0.029  0.020  0.048  0.999  0.000   0.000   0.000
bread  -0.073 -0.109 -0.065 -0.085  0.002  0.931   0.000   0.000
hunger -0.062 -0.049 -0.025 -0.011  0.067  0.040   1.010   0.000
thirst  0.002  0.003 -0.002  0.005  0.032 -0.002   0.458   0.875 

Sentence similarities (dot product):

       S1     S2     S3     S4     S5     S6
S1  1.031  0.000  0.000  0.000  0.000  0.000
S2  0.050  1.095  0.000  0.000  0.000  0.000
S3  0.685  0.044  1.044  0.000  0.000  0.000
S4  0.100  0.084  0.245  1.098  0.000  0.000
S5  0.299 -0.023  0.251  0.714  1.121  0.000
S6 -0.050 -0.029 -0.058  0.531  0.204  1.058<jupyter_text>Extracting fillers and roles from frames"[...]frame is convolved with the approximate inverse of the role and the result is cleaned up by choosing the most similar vector in the item memory."(recreating Table 8 from Plate)<jupyter_code># agent of s1
agent = sentences[1].decode(tokens['agent_eat'])
results = hrr.getClosest(agent, T)
print("agent of s1")
for k in sorted(results.items(), key=lambda x: x[1], reverse=True):
    print(k)
#agent of s1(w/o frame label)
agent = sentences[1].decode(tokens['agent'])
results = hrr.getClosest(agent, T)
print("agent of s1(w/o frame label)")
for k in sorted(results.items(), key=lambda x: x[1], reverse=True):
    print(k)
# object of s1
agent = sentences[1].decode(tokens['object_eat'])
results = hrr.getClosest(agent, T)
print("object of s1")
for k in sorted(results.items(), key=lambda x: x[1], reverse=True):
    print(k)
# agent of s2
agent = sentences[2].decode(tokens['agent_cause'])
results = hrr.getClosest(agent, T)
print("agent of s2")
for k in sorted(results.items(), key=lambda x: x[1], reverse=True):
    print(k)
# object of s2
agent = sentences[2].decode(tokens['object_cause'])
results = hrr.getClosest(agent, T)
print("object of s2")
for k in sorted(results.items(), key=lambda x: x[1], reverse=True):
    print(k)
# agent of object of s2
agent = sentences[2].decode(tokens['object_cause'].encode(tokens['agent_eat']))
results = hrr.getClosest(agent, T)
print("agent of object of s2")
for k in sorted(results.items(), key=lambda x: x[1], reverse=True):
    print(k)
# object of object of s2
agent = sentences[2].decode(tokens['object_cause'].encode(tokens['object_eat']))
results = hrr.getClosest(agent, T)
print("object of object of s2")
for k in sorted(results.items(), key=lambda x: x[1], reverse=True):
    print(k)
# role of 'john' in s4
agent = sentences[4].decode(tokens['john'])
results = hrr.getClosest(agent, T)
print("role of 'john' in s4")
for k in sorted(results.items(), key=lambda x: x[1], reverse=True):
    print(k)
# role of 'john' in s5
agent = sentences[5].decode(tokens['john'])
results = hrr.getClosest(agent, T)
print("role of 'john' in s5")
for k in sorted(results.items(), key=lambda x: x[1], reverse=True):
    print(k)
#Sequences
seq = 'abc'

#Stacks
#Chunking sequences
#Variable binding
#Frames(slot/filler)
#recursive frames (WHERE HRRs SHINE)
    #commutativity limitation of HRRs

# addition memory capacity
n=64 #elements in each addition memory vector
k=3 #distinct items to be stored in the memory trace
m=100 #total possible items 
samples = 100
out_samples = np.empty((samples, m-k))
in_samples = np.empty((samples, k))
#finding optimal threshold for 
for s in range(samples):
    M = {i:hrr.AdditionMemory(n_dims=64) for i in range(m)}
    mems_in_trace = np.random.choice(range(m), 3, replace=False)
    trace = reduce(lambda x,y: x+y, [M[i] for i in mems_in_trace]) #sum() does not work on trace here--why?
    out_samples[s] = [M[o]*trace for o in range(m) if o not in mems_in_trace]
    in_samples[s] = [M[i]*trace for i in mems_in_trace]
outs = out_samples.reshape(samples*(m-k),1)
ins = in_samples.reshape(samples*k,1)

#poor man's optimization with probabilistic constraints
#finding the threshold
from scipy.optimize import minimize_scalar


def classifyDummyItems(t, n=64, m=100, k=3, mode='dist', iterations=100):
    data = np.empty(100)
    if mode=='dist': #with signal drawn from analytical distributions
        for i in range(iterations):
            in_data = np.random.normal(1, math.sqrt((k+1)/n), k) #dp of an item in memory
            out_data = np.random.normal(0, math.sqrt(k/n), (m-k)) #dp of an item not in memory
            data[i] = 1-((np.count_nonzero(in_data > t)/len(in_data))**k) * ((np.count_nonzero(out_data < t)/len(out_data))**(m-k)) #incorrects/total
    elif mode=='items': #with signal computed from items
        for i in iterations:
            trace = sum([hrr.AdditionMemory(n_dims = n) for j in k])
            
    #score this t's performance
    return np.mean(data)

dist_min = minimize_scalar(lambda x: classifyDummyItems(x, n=64, m=100, k=3, mode='dist'), 0.5, bounds=(-2, 2), method='bounded')

#end: plot s_a dist and histogram, s_r dist and histogram, threshold, 
plt.figure(figsize=(10, 10))

x = np.linspace(-2, 2, 500)
n_o, bins_o, patches_o = plt.hist(outs, bins='auto', density=True, alpha=0.5)
n_i, bins_i, patches_i = plt.hist(ins, bins='auto', density=True, alpha=0.5)
plt.plot(x,mlab.normpdf(x, 1, math.sqrt((k+1)/n)), label='accept') 
plt.plot(x,mlab.normpdf(x, 0, math.sqrt(k/n)), label='reject')
plt.axis([-1.5, 2.5, 0, 2])
plt.xlabel('dot product with trace', fontsize=24)
plt.ylabel('probability', fontsize=24)
plt.xticks(fontsize=18)
plt.yticks(fontsize=18)
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>The above took four items and gave them unique HRR "signatures" as representations. Two pairs of items' HRRs were convolved together into new HRRs, and these resulting traces were composed together. This final HRR represents two pairs of items.By taking any item's HRR, and decoding the composed trace, we can return an HRR representing the item's pair. Comparing the decoded HRR to those in an associative memory   A few sanity checks- verify elements of convolution- commutativity of convolution- xi and eta (noise terms) compared to theoretical values- decoding terms not in a trace should result in noise?- TODO: means and variances of commmon convolution ops (cf Table 1)<jupyter_code>#elements of our HRRs
Q = 10
dim = 512
samples = np.empty((Q, dim))
for q in range(Q):
    samples[q] = hrr.HRR().values
samples = samples.reshape((Q*dim,1))
                
plt.figure(figsize=(20, 10))
sigma = math.sqrt(1/dim)
x = np.linspace(-3*sigma, 3*sigma, 50)
# n, bins, patches = plt.hist(samples, bins=50, density=True, range=(-3*sigma, 3*sigma))
n, bins, patches = plt.hist(samples,bins='auto', density=True)
plt.axis([-3*sigma, 3*sigma, 0, 15])
plt.plot(x,mlab.normpdf(x, 0, sigma))
plt.xlabel('element value', fontsize=24)
plt.ylabel('probability', fontsize=24)
plt.xticks(fontsize=18)
plt.yticks(fontsize=18)
plt.show()

#is encoding working as intended?
# C encoding X (c.encode(x))
C = ['c0', 'c1', 'c2']
X = ['x0', 'x1', 'x2']
n = len(C)

#circular convolution (HRR)
#from hrr.py class HRR method convolve() (aka encode):
hrr_terms = [' + '.join([C[k%n]+'*'+X[(j-k)%n] for k in range(0,len(X))]) for j in range(0, len(C))]

#aperiodic convolution
J = range(-(n-1), n)
K = range(int(-(n-1)/2), int((n-1)/2)+1)
aper_terms = []
for j in J:
    tmp_sum = []
    for k in K:
        # print('j:',j,'k:',k)
        item_ind = k+int((n-1)/2)
        self_ind = j-k+int((n-1)/2) 
        if self_ind >= 0 and self_ind < n and item_ind >= 0 and item_ind < n:
            # print('ITEM:',item_ind)
            # print('SELF:',self_ind)
            tmp_sum.append(X[item_ind] + '*' + C[self_ind])
    aper_terms.append(' + '.join(tmp_sum))
    
#truncated aperiodic convolution
J = range(int(-(n-1)/2), int((n-1)/2)+1)
K = range(int(-(n-1)/2), int((n-1)/2)+1)
trunc_terms = []
for j in J:
    tmp_sum = []
    for k in K:
        item_ind = k+int((n-1)/2)
        self_ind = j-k+int((n-1)/2)
        if self_ind >= 0 and self_ind < n and item_ind >= 0 and item_ind < n:
            tmp_sum.append(X[item_ind] + '*' + C[self_ind])
    trunc_terms.append(' + '.join(tmp_sum))

print(C, ' encoding ', X, ' : \n')
print('HRR:\n','\n '.join(hrr_terms), '\n')
print('Aperiodic:\n', '\n '.join(aper_terms), '\n')
print('Truncated aperiodic:\n', '\n '.join(trunc_terms), '\n')

#is decoding working as intended?
t = hrr_terms<jupyter_output><empty_output><jupyter_text>Is the HRR convolution commutative?<jupyter_code>#take two random HRRs h1 and h2, 
#convolve them--h1.encode(h2) and h2.encode(h1)--
# and compare the resulting traces
h1 = hrr.HRR()
h2 = hrr.HRR()

t1 = h1.encode(h2)
t2 = h2.encode(h1)

#since we're using floating point, compare values with some (very small) tolerance
print(np.allclose(np.array(t1.values),np.array(t2.values)))
# print(np.array(t1.values)-np.array(t2.values))<jupyter_output><empty_output><jupyter_text>Do xi and eta follow their theoretical distributions? How do they vary wrt their assumed distributions as a function of HRR dimension? Plate says n=16 is sufficient for assuming normality.<jupyter_code># #finding eta and xi via least squares
# for q in range(Q):
#     for p in range(q, Q):
#         #since convolution is commutative, only compare one set of pairs. And ignore self-to-self comparisons
#         r_ = M[p].decode(M[p].encode(M[q])) # get a noisy version of our vector q
#         r = np.array(r_.values) 
#         a_ = np.array([M[q].values])
#         a = np.eye(N)
#         b = np.array(M[q].values)-r #"dependent" values, b = r[i] - q[i] : the difference between expected and actual element values
# #         print(b)
# #         vals = np.linalg.(a,b)[0] #
# #         xi[q][p] = vals[0]
#         eta[q][p] = b    
        
# how do eta and xi change with increasing n?
plt.figure(figsize=(25, 10))

#HRR dimension
Ns=[4, 8, 16, 32, 64]
#Number of samples
Q=10000
#store a label for each HRR
# M={q:hrr.HRR(n_dims=N) for q in range(Q)}
#data arrays
# xi = np.empty((Q,Q))
# xi[:] = np.nan #to not skew stats later
# eta = np.empty((Q,Q,N)) #N is i for eta_i (element i in noise vector)
# eta[:] = np.nan

for N in Ns:
    xi = np.empty((Q,1))
    eta = np.empty((Q,N))
    eta_mean = np.empty((Q,1))
    eta_one = np.empty((Q,1))
    elements = np.empty((Q, N*2))
    for q in range(Q):
        h1 = hrr.HRR(n_dims=N)
        h2 = hrr.HRR(n_dims=N)
        h = h1.decode(h1.encode(h2))
    #     xi[q] = np.sum(np.square(np.random.normal(0, math.sqrt(1/N), (N,1))))-1
        xi[q] = sum(map(lambda x: x**2, h1.values))-1
        eta[q] = np.array(h.values) - (np.array(h2.values)*(1+xi[q]))
        eta_mean[q] = np.nanmean(eta[q])
        eta_one[q] = eta[q][0]
        elements[q] = h1.values + h2.values
    
    #xi plot
    plt.subplot(2,len(Ns),(Ns.index(N)+1))
    sigma_xi = math.sqrt(2/N)
    x = np.linspace(-3*sigma_xi, 3*sigma_xi, 50)
    n, bins, patches = plt.hist(xi, bins='auto', density=True, range=(-3*sigma_xi, 3*sigma_xi))
    y_axis = max(n)
    plt.axis([-3*sigma_xi, 3*sigma_xi, 0, y_axis])
    plt.plot(x,mlab.normpdf(x, 0, sigma_xi), label='N(mu=0,sigma=sqrt(2/N)')
    plt.legend()
    plt.title('N = '+str(N))
    plt.xlabel('xi', fontsize=24)
    plt.ylabel('probability', fontsize=24)
    plt.xticks(fontsize=18)
    plt.yticks(fontsize=18)

    #eta plot
    plt.subplot(2,len(Ns),(Ns.index(N)+6))
    sigma_eta = math.sqrt((N-1)/(N**2))
    x = np.linspace(-3*sigma_eta, 3*sigma_eta, 50)
    # vals = eta.reshape(eta.size, 1)
    vals = eta_one
    n, bins, patches = plt.hist(vals, bins='auto', density=True, range=(-3*sigma_eta, 3*sigma_eta))
    y_axis = max(n)
    plt.axis([-3*sigma_eta, 3*sigma_eta, 0, y_axis])
    plt.plot(x,mlab.normpdf(x, 0, sigma_eta), label='N(mu=0,sigma=sqrt((N-1)/(N**2))')
    plt.legend()
    plt.title('N = '+str(N))
    plt.xlabel('eta', fontsize=16)
    plt.ylabel('probability', fontsize=16)
    plt.xticks(fontsize=14)
    plt.yticks(fontsize=14)


plt.show()<jupyter_output><empty_output><jupyter_text>The distributions of dot products of some common convolution expressions  (Plate 1995, Table 1)<jupyter_code>samples = 10
data = np.empty((samples, 10))
for s in range(samples):
    a,b,c,d = [hrr.HRR()]*4
    data[s]=[a*a, a*b, a*(a.encode(b)), a*(b.encode(c)), (a.encode(a))*(a.encode(a)),\
             (a.encode(b))*(a.encode(b)), (a.encode(b))*(a.encode(a)), \
             (a.encode(b))*(a.encode(c)), (a.encode(b))*(c.encode(c)), \
             (a.encode(b))*(c.encode(d))]

dists={
    0:lambda x: (1, 2/x),
    1:lambda x: (0, 1/x),
    2:lambda x: (0, (((2*x)+1))/(x**2)),
    3:lambda x: (0, 1/x),
    4:lambda x: ((((2*x)+2))/(x), (((40*x)+112))/(x**2)),
    5:lambda x: (1, (((6*x)+4))/(x**2)),
    6:lambda x: (0, (((6*x)+18))/(x**2)),
    7:lambda x: (0, (((2*x)+2))/(x**2)),
    8:lambda x: (0, (((2*x)+2))/(x**2)),
    9:lambda x: (0, 1/x),
}

# for i in range(1):
#     plt.subplot(5,2,i)
#     plt.hist(data[:][i], density=True)

    
    

# sigma_xi = math.sqrt(2/N)
# x = np.linspace(-3*sigma_xi, 3*sigma_xi, 50)
# n, bins, patches = plt.hist(xi, bins='auto', density=True, range=(-3*sigma_xi, 3*sigma_xi))
# y_axis = max(n)
# plt.axis([-3*sigma_xi, 3*sigma_xi, 0, y_axis])
# plt.plot(x,mlab.normpdf(x, 0, sigma_xi))<jupyter_output><empty_output><jupyter_text>Demos<jupyter_code>#HRR
#evaluate percent correct retrievals as a function of encoded pairs

#dimension (number of elements) of HRR to evaluate
N = [8, 16, 32, 64, 128, 256]
#number of pairs to encode in a single HRR
P = [1]+list(range(5,26,5))
#number of samples to take of each (dimension, NumPairs) pair
samples = 25


data = np.empty((len(N), len(P), samples))

for n in range(len(N)):
    for p in range(len(P)):
        for s in range(samples):
            X = {(str(i)+'x'):hrr.HRR(n_dims=N[n]) for i in range(P[p])}
#             print('X', X)
            Y = {(str(i)+'y'):hrr.HRR(n_dims=N[n]) for i in range(P[p])}
#             print('Y', Y)
            M = dict(list(X.items()) + list(Y.items()))
#             print('M', M)
            pairs = [X[str(i)+'x'].encode(Y[str(i)+'y']) for i in range(P[p])]
            trace = reduce((lambda x,y: x+y), pairs)
            score = 0
            for h in range(P[p]):
                ans = list(hrr.getClosest(item=trace.decode(M[str(h)+'x']), memoryDict=M, howMany=1))[0]
                if ans == str(h)+'y':
#                     print(str(h)+'x', '->', ans, '?=', str(h)+'y', '  +1')
                    score+=1
#                 else:
#                     print(str(h)+'x', '->', ans, '?=', str(h)+'y', '  +0')
                ans = list(hrr.getClosest(item=trace.decode(M[str(h)+'y']), memoryDict=M, howMany=1))[0]
                if ans == str(h)+'x':
#                     print(str(h)+'y', '->', ans, '?=', str(h)+'x', '  +1')
                    score+=1
#                 else:
#                     print(str(h)+'y', '->', ans, '?=', str(h)+'x', '  +0')
#             print(score/P[p])
            data[n][p][s]= score / (P[p]*2)

a = plt.figure(figsize=(20, 10))
x=list(P)

means = {N[n]:np.mean(data[n],axis=1) for n in range(len(N))}
SDs = {N[n]:np.std(data[n],axis=1) for n in range(len(N))}
# print(SDs)

for n in N:
    plt.errorbar(x, means[n], SDs[n], label=str(n),) 

# plt.plot(np.linspace(start=1,stop=max(P),retstep=1))
plt.axis([0, max(P)+1, 0, 1])
plt.xlabel('Number of stored pairs', fontsize=24)
plt.ylabel('Probability of correct decoding', fontsize=24)
plt.xticks(x,fontsize=18)
plt.yticks(fontsize=18)
L=plt.legend(title='Number of vector elements', fontsize=18)
plt.setp(L.get_title(),fontsize=18)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17730745587072977, 'fraction_numerical': 0.04786159655747783, 'mean_word_length': 3.8313534153585067, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 44, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '233543', 'license': 'MIT', 'n_tokens_mistral': 9377, 'n_tokens_neox': 8605, 'n_words': 2301, 'path': 'hrr_demo.ipynb', 'repo_name': 'tah0/hrr', 'size': '22774'}