<jupyter_start><jupyter_text>Exploring Numpy Array Broadcasting rulesKey idea = The Broadcasting Rule> In order to broadcast, the size of the trailing axes for both arrays in an operation must either be the same size or one of them must be one.see:  - https://numpy.org/doc/stable/user/basics.broadcasting.html - https://numpy.org/devdocs/user/theory.broadcasting.html<jupyter_code>import numpy as np

X = np.array(np.mat(
"""
0 1 2;
1 1 1;
2 2 2;
3 3 3
"""))
print(X.shape)
X
v = np.array([10,11,12])
print(v.shape)
v
print('X.shape', X.shape)
print('v.shape', v.shape)
X + v
# alternatively make explicit 1
vv = np.reshape(v, (1,3))
print(vv.shape)
vv
print('X.shape', X.shape)
print('vv.shape', vv.shape)
X + vv
Y = np.array(np.mat(
"""
5 5 5;
6 6 6;
7 7 7;
8 8 8;
9 9 9
"""))
Y<jupyter_output><empty_output><jupyter_text>One for loop approach<jupyter_code>diffs1 = np.zeros( (X.shape[0], Y.shape[0]) )

for i in range(X.shape[0]):
    x_test = X[i,:]
    # diffs1[i,:] = -1*np.sum(Y - x_test, axis=1)
    diffs1[i,:] = np.sqrt(np.sum(np.square(Y - x_test), axis=1))


diffs1
# check diffs1 against naive method (two for loops)
for i in range(X.shape[0]):
    for j in range(Y.shape[0]):
        expected_ij = np.sqrt(np.sum(np.square(X[i,:]-Y[j,:])))
        assert np.isclose(diffs1[i,j], expected_ij), f'mismatch at i={i}, j={j}'
print('All close')

Y - x_test
np.sqrt( np.sum(np.square(Y - x_test), axis=1) )
dists_row = np.sum(np.square(Y - x_test), axis=1)
dists_row<jupyter_output><empty_output><jupyter_text>Zero for loops approach (memory inefficient)<jupyter_code># need to augment X so it has a trailing-1 dimension
XX = X[:, :, np.newaxis]
print(XX.shape)
# XX
## Another way to do the same thing w/o using np.newaxis
# XX = X.reshape(*X.shape, 1)
# print(XX.shape)
# XX
# compute the difference
print('XX.shape', XX.shape)
print('Y.T.shape', Y.T.shape)
print('(XX - Y.T).shape', (XX - Y.T).shape)
XX - Y.T
# reduce using sum along middle dimension
diffs0m = np.sqrt(np.sum(np.square(XX - Y.T), axis=1))
diffs0m
# check diffs0m against naive method (two for loops)
for i in range(X.shape[0]):
    for j in range(Y.shape[0]):
        expected_ij = np.sqrt(np.sum(np.square(X[i,:]-Y[j,:])))
        assert np.isclose(diffs0m[i,j], expected_ij), f'mismatch at i={i}, j={j}'
print('All close')
# note this is very memory inefficient
# float64 or int64 take up 8 bytes (64 bits)
# and we have an intermediate result array of shape (4, 3, 5) 
# so this way would require allocating memory for intemrediate step:
print('Using this memory inefficient way in assignment would require')
print(8*(4*3*5), 'bytes')
# float64 or int64 take up 8 bytes (64 bits)
# and we have an intermediate result array of shape (500, 3072, 5000) 
# so this way would require:
print('Using this memory inefficient way in assignment would require')
print(8*(500*3072*5000)/1024/1024/1024, 'GB')<jupyter_output>Using this memory inefficient way in assignment would require
57.220458984375 GB<jupyter_text>Zero for loops approach (fast)Rewrite using ![rewrite trick](https://miro.medium.com/max/776/1*ZU5JN1yhEr6ga_N772gInA.png)<jupyter_code>X
XSQ = np.sum(np.square(X), axis=1)[:, np.newaxis]
XSQ
YSQ = np.sum(np.square(Y), axis=1)[np.newaxis,:]
YSQ
XSQ + YSQ
diffs0 = np.sqrt(XSQ + YSQ - 2*X.dot(Y.T))
diffs0
# check diffs0 against naive method (two for loops)
for i in range(X.shape[0]):
    for j in range(Y.shape[0]):
        expected_ij = np.sqrt(np.sum(np.square(X[i,:]-Y[j,:])))
        assert np.isclose(diffs0[i,j], expected_ij), f'mismatch at i={i}, j={j}'
print('All close')
x = np.arange(9)
splits = np.array_split(x, 4)
splits
np.concatenate((splits[0], splits[2]))
accuracies = np.zeros(3)
accuracies<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16228540772532188, 'fraction_numerical': 0.04157725321888412, 'mean_word_length': 4.5081240768094535, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 17, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '206848', 'license': 'MIT', 'n_tokens_mistral': 1586, 'n_tokens_neox': 1462, 'n_words': 403, 'path': 'assignment1/ExploreNumpyBroadcasting.ipynb', 'repo_name': 'ivanistheone/cs231n', 'size': '3728'}