<jupyter_start><jupyter_text>Numerical operations with Numpy 3.1 **Broadcasting** 3.2 **Array shape manipulation**  3.3 **Sorting data**  **Summary** **Exercises**<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline<jupyter_output><empty_output><jupyter_text>3.1 Broadcasting Operations-  We just covered basic operations (add, multiple, square etc) such are element-wise but that works on arrays of same size- **Broadcasting** comes handy when we are dealing with different shapes. This time, we'll explore a more advanced concept in numpy called broadcasting. - The term broadcasting describes how numpy treats arrays with different shapes during arithmetic operations. Subject to certain constraints, *the smaller array is "broadcast" across the larger array so that they have compatible shapes*. - Broadcasting provides a means of **vectorizing array operations** so that looping occurs in C instead of Python. It does this without making needless copies of data and usually leads to efficient algorithm implementations. There are also cases where broadcasting is a bad idea because it leads to inefficient use of memory that slows computation.- In this little tutorial we will provide a gentle introduction to broadcasting with numerous examples ranging from simple to involved. - We will also go through a few examples of when to and when not to use boradcasting. This example below shows how broadcasting works  So, lets start taking baby steps...Here an element-wise multiplication occurs since the two arrays are of same shape<jupyter_code>e = np.array([1.0, 2.0, 3.0])
f = np.array([2.0, 2.0, 2.0])
e*f<jupyter_output><empty_output><jupyter_text>Hint / Try it?What would have happened if `f = np.array([2.0, 2.0])`. would it still multiply?<jupyter_code># But if it was like this

e = np.array([1.0, 2.0, 3.0])
f = 2.0
e*f<jupyter_output><empty_output><jupyter_text>What happened hereThis is the most simplest example on numpy broadcasting where an array and a scalar values were combined in an operation.so it kind of *stechted in the row direction*! The scalar **f** is stretched to become an array of with the same shape as **e** so the shapes are compatible for element-by-element multiplication.** So what are the rules then?**- They must either be equal / same shapeOR- One of them must be 1, like f was above<jupyter_code># Typical broadcasting in practice
g = np.array([[ 0.0, 0.0, 0.0], [10.0,10.0,10.0],[20.0,20.0,20.0],[30.0,30.0,30.0]])
g       
h = np.array([1.0, 2.0, 3.0])
h
g + h<jupyter_output><empty_output><jupyter_text>What happened above?A 2-D (two-dimensional) array multiplied by 1-D (one-dimensional) array. It got stretched in the column direction so as to match the elements of the 2D array columns.Would the same be possible for different shapes? Does broadcasting magically understands and fixes our assumptions?Let's take a look...<jupyter_code>g = np.array([[ 0.0, 0.0, 0.0], [10.0,10.0,10.0],[20.0,20.0,20.0],[30.0,30.0,30.0]])
i = np.array([0.0, 1.0, 2.0, 3.0])
g+i<jupyter_output><empty_output><jupyter_text>We had a mismatch...Explanation: When the trainling dimensions of the arrays are different as you saw above, then broadcasting will fail making it impossible to align the values in the rows of the first array with the elements of the second array for an **element-by-element** addition or multiplication. Also, is there a way to do this in one line of codeTip: look up more into np.tile and np.arange<jupyter_code>a = np.tile(np.arange(0, 40, 10), (3, 1))
a = a.T # transpose this
a
b = np.array([0, 1, 2])
b<jupyter_output><empty_output><jupyter_text>Now, we add these two<jupyter_code>a + b<jupyter_output><empty_output><jupyter_text>So you see that broadcasting was applied magically...Ask yourself, why couldn't we add original `a` and `b` ?Note, original a was:```pythonarray([[ 0, 10, 20, 30],       [ 0, 10, 20, 30],       [ 0, 10, 20, 30]])```<jupyter_code>c = np.ones((5, 6))
c<jupyter_output><empty_output><jupyter_text>Let's assign an array of dimension 0 to an array of dimension 1<jupyter_code>c[0] = 2
c
d = np.arange(0, 30, 10)
d
d.shape
d = d[:, np.newaxis] # Here we add a new axis and make it a 2D array
d.shape
a + d<jupyter_output><empty_output><jupyter_text>Another example on broadcastingLet’s construct an array of distances (in miles) between cities of Route 66: Chicago, Springfield, Saint-Louis, Tulsa, Oklahoma City, Amarillo, Santa Fe, Albuquerque, Flagstaff and Los Angeles.<jupyter_code>mileposts = np.array([0, 198, 303, 736, 871, 1175, 1475, 1544, 1913, 2448])
distance_array = np.abs(mileposts - mileposts[:, np.newaxis])
distance_array<jupyter_output><empty_output><jupyter_text>Another exampleA lot of grid-based or network-based problems can also use broadcasting. For instance, if we want to compute the distance from the origin of points on a 10x10 grid, we can do<jupyter_code>x, y = np.arange(5), np.arange(5)[:, np.newaxis]
distance = np.sqrt(x**2 + y**2)
distance<jupyter_output><empty_output><jupyter_text>Or in color...<jupyter_code>plt.pcolor(distance)
plt.colorbar
# Note : The numpy.ogrid function allows to directly create vectors
# x and y of the previous example
x, y = np.ogrid[0:5, 0:5]
x, y
x.shape, y.shape<jupyter_output><empty_output><jupyter_text>`np.ogrid` is quite useful as soon as we have to handle computations on a grid. While on other hand, `np.mgrid` directly provides matrices full of indices for cases where we can't or maybe don't want to benefit from broadcasting.<jupyter_code>x, y = np.mgrid[0:4, 0:4]
x
y<jupyter_output><empty_output><jupyter_text>A bit on Vector quantization or VQA simple way to understand bradcasting is with this real world situation. The basic operatio in VQ finds the closest point in a set of points, called $codes$ in VQ speak, to a given point, called the `observation`.In the 2D example below, the value in an $observation$ describe the weight and height of an athlete to be classified. The $codes$ represent different classes of athletes such as dancer, runner, swimmer an so on.Finding the closest point requires calculating the distance between `observation` and each of the `codes`.The shortest distance provides the best match. Here in this example, `codes[0]` is the closest class indicating that the athlete is likely a basketball player.<jupyter_code>from numpy import array, argmin, sqrt, sum
observation = array([111.0, 188.0])
codes = array([[102.0, 203.0],
               [132.0, 193.0],
                [45.0, 155.0],
                [57.0, 173.0]])
# This is how broadcast happens
difference = codes - observation
distance = sqrt(sum(difference**2, axis=-1))
nearest = argmin(distance)
nearest<jupyter_output><empty_output><jupyter_text>The basic operation of vector quantization calculates the distance between an object to be classified, the black square, and multiple known codes, the gray circles. In the very basic case, the codes represent classes.  A more advanced example  `@article{scikit-learn, title={Scikit-learn: Machine Learning in {P}ython}, author={<NAME>. and <NAME>. and <NAME>. and <NAME>.         and <NAME>. and <NAME>. and <NAME>. and <NAME>.         and <NAME>. and <NAME>. and <NAME>. and <NAME>. and         Cournape<NAME>.}, journal={Journal of Machine Learning Research}, volume={12}, pages={2825--2830}, year={2011}}`<jupyter_code># A more complex example
import numpy as np
import scipy as sp
import matplotlib.pyplot as plt

from sklearn import cluster


try:  # SciPy >= 0.16 have face in misc
    from scipy.misc import face
    face = face(gray=True)
except ImportError:
    face = sp.face(gray=True)

n_clusters = 5
np.random.seed(0)

X = face.reshape((-1, 1))  # We need an (n_sample, n_feature) array
k_means = cluster.KMeans(n_clusters=n_clusters, n_init=4)
k_means.fit(X)
values = k_means.cluster_centers_.squeeze()
labels = k_means.labels_

# create an array from labels and values
face_compressed = np.choose(labels, values)
face_compressed.shape = face.shape

vmin = face.min()
vmax = face.max()

# original face
plt.figure(1, figsize=(3, 2.2))
plt.imshow(face, cmap=plt.cm.gray, vmin=vmin, vmax=256)

# compressed face
plt.figure(2, figsize=(3, 2.2))
plt.imshow(face_compressed, cmap=plt.cm.gray, vmin=vmin, vmax=vmax)

# equal bins face
regular_values = np.linspace(0, 256, n_clusters + 1)
regular_labels = np.searchsorted(regular_values, face) - 1
regular_values = .5 * (regular_values[1:] + regular_values[:-1])  # mean
regular_face = np.choose(regular_labels.ravel(), regular_values, mode="clip")
regular_face.shape = face.shape
plt.figure(3, figsize=(3, 2.2))
plt.imshow(regular_face, cmap=plt.cm.gray, vmin=vmin, vmax=vmax)

# histogram
plt.figure(4, figsize=(3, 2.2))
plt.clf()
plt.axes([.01, .01, .98, .98])
plt.hist(X, bins=256, color='.5', edgecolor='.5')
plt.yticks(())
plt.xticks(regular_values)
values = np.sort(values)
for center_1, center_2 in zip(values[:-1], values[1:]):
    plt.axvline(.5 * (center_1 + center_2), color='b')

for center_1, center_2 in zip(regular_values[:-1], regular_values[1:]):
    plt.axvline(.5 * (center_1 + center_2), color='b', linestyle='--')

plt.show()<jupyter_output><empty_output><jupyter_text>3.2 Array Shape Manipulation Flattening<jupyter_code>a = np.array([[1, 2, 3], [4, 5, 6]])
a.ravel() 
"""
A 1-D array, containing the elements of the input, is returned.  A copy is
    made only if needed.
    Do help(np.ravel) to learn more
"""
a.T<jupyter_output><empty_output><jupyter_text>Reshaping<jupyter_code>a.shape
a.reshape(-1)
b = a.ravel()
b
b = b.reshape((2, 3))
b
# Which is same as ...
a.reshape(2, -1)

# Note:  ndarray.reshape may return a view (cf help(np.reshape))), or copy
b[0, 0] = 99
a
# Reshape also returns a copy, take a look
a = np.zeros((3, 2))
b = a.T.reshape(3 * 2)
b [0] = 9
a<jupyter_output><empty_output><jupyter_text>Memory layout of a numpy array- [Here's a good example of how it works](https://eli.thegreenplace.net/2015/memory-layout-of-multi-dimensional-arrays/)-<jupyter_code>x = np.random.rand(2, 2)
x.data
x.__array_interface__['data']
x[0].__array_interface__['data']
x[0,:].__array_interface__['data']
x[1,:].__array_interface__['data']
x[0,0].__array_interface__['data']<jupyter_output><empty_output><jupyter_text>3.3 Sorting Data *Function*`sort (arr, axis=-1, kind='quick', order=None)`*Method*`arr.sort (axis=-1, kind='quick', order=None)`<jupyter_code># Sorting along an axis. see what happens?
a = np.array([[1, 4, 3], [3, 1, 3]])
b = np.sort(a, axis=1)
print(b)
# In-place sort
a.sort(axis=1)
print(a)
# Sorting with fancy indexing
a = np.array([5, 4, 6, 1])
x = np.argsort(a)
x
# Finding minima and maxima
b = np.array([3,5,2,6])
b_max = np.argmax(b)
b_min = np.argmin(b)
print(b_max)
print(b_min)<jupyter_output>3
2<jupyter_text>Some Exercises  😅  1. Array manipulationsCreate this 2D array (without typing manually)`[[1,  7, 12], [2,  8, 13], [3,  9, 14], [4, 10, 15], [5, 11, 16]]`  2.    Fun Exercises: Challenge questions1. Try in-place, out_of_place sorting2. Create arrays with different dtypes and sort them.3. Use all or array_equal to see what it returns4. Use np.random.shuffle to create a more sortable input5. Combine ravel, sort and reshape in one6. Look at the `axis` keyword for `sort` and rewrite the previous exercise<jupyter_code>a = np.arange(25).reshape(5, 5)
a
help(np.sum)
help(np.matrix.sum)
np.sum([1.0, 1.5])
np.sum([1.0, 0.4, 0.5, 0.6], dtype=np.int32)
np.sum([[0, 2], [0, 6]])
np.sum([[0, 2], [0, 6]], axis=0)
np.sum([[0, 2], [0, 6]], axis=1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1276558841384095, 'fraction_numerical': 0.038938513572109965, 'mean_word_length': 4.895705521472393, 'pattern_counts': {'":': 0, '<': 93, '<?xml version=': 0, '>': 94, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '20', 'dir': 'jupyter-structured-clean-dedup', 'id': '95752', 'license': 'Apache-2.0', 'n_tokens_mistral': 4258, 'n_tokens_neox': 3926, 'n_words': 1364, 'path': 'numpy/SciPy Numpy Extended Lectures Part 3 - Broadcasting, Shape Manipulation and Sorting Data.ipynb', 'repo_name': 'HiteshDhola/Artificial-Intelligence-Deep-Learning-Machine-Learning-Tutorials', 'size': '11635'}