<jupyter_start><jupyter_text>**Numpy Tutorial**Numpy is a library for the Python programming language, adding support for large, multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays.While Python is slower in execution as compared to other languages like Fortran during looping, Numpy fastens Python’s operations by converting repetitive code to compiled form.A Numpy Array is very similar to a Python list with a special condition that all of its elements must be of the same type.<jupyter_code>#Import Numpy Library
import numpy as np<jupyter_output><empty_output><jupyter_text>**1-Creating a Vector**<jupyter_code>#Load Library
import numpy as np

#Create a vector as a Row
vector_row = np.array([1,2,3])

#Create vector as a Column
vector_column = np.array([[1],[2],[3]])

# vector_row
vector_column<jupyter_output><empty_output><jupyter_text>**2-Creating a Matrix**We Create a 2-D Array in Numpy and call it a Matrix. It contains 2 rows and 3 columns.<jupyter_code>#Load Library
import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6]])
print(matrix)

# Find if a value exists in the array...Returns true if value exists else returns false
print(2 in matrix)
# Array of zeros
print(np.zeros(10))

# Array of ones with type int
print(np.ones(10, dtype=int))

# Range of numbers
rangeArray = np.array(range(10), int)
print(rangeArray)<jupyter_output>[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[1 1 1 1 1 1 1 1 1 1]
[0 1 2 3 4 5 6 7 8 9]<jupyter_text>**3-Creating a Sparse Matrix**Sparse Matrices store only non zero elements and assume all other values will be zero, leading to significant computational savings.<jupyter_code>#Load Library
import numpy as np
from scipy import sparse

#Create a Matrix
matrix = np.array([[0,0],[0,1],[3,0]])
print(matrix)

#Create Compressed Sparse Row(CSR) matrix
matrix_sparse = sparse.csr_matrix(matrix)
print(matrix_sparse)<jupyter_output>[[0 0]
 [0 1]
 [3 0]]
  (1, 1)	1
  (2, 0)	3<jupyter_text>**4-Selecting Elements**When you need to select one or more element in a vector or matrixWe can index the elements. Index in numpy array also starts with 0, so matrix[0] refers to the first element that is 1. We can also define a range such as [:2] which prints all values at indices 0 to 1. Creating such an array is highly useful because of its immense potential just like simply checking for an element in the array.<jupyter_code>#Load Library
import numpy as np

#Create a vector as a Row
vector_row = np.array([ 1,2,3,4,5,6 ])

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix)

#Select 3rd element of Vector
print(vector_row[2])

#Select 2nd row 2nd column
print(matrix[1,1])

#Select all elements of a vector
print(vector_row[:])

#Select everything up to and including the 3rd element
print(vector_row[:3])

#Select the everything after the 3rd element
print(vector_row[3:])

#Select the last element
print(vector_row[-1])

#Select the first 2 rows and all the columns of the matrix
print(matrix[:2,:])

#Select all rows and the 2nd column of the matrix
print(matrix[:,1:2])
#Selects all rows from row0 onwards and all columns from column1 onwards
arr2=arr1[0:,1:]
arr2
# Select the 2nd Row & 3rd Column Value
matrix[1][2]<jupyter_output><empty_output><jupyter_text>**5-Describing a Matrix**When you want to know about the shape size and dimensions of a Matrix.<jupyter_code>import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])

#View the Number of Rows and Columns
print(matrix.shape)

#View the number of elements (rows*columns)
print(matrix.size)

#View the number of Dimensions(2 in this case)
print(matrix.ndim)<jupyter_output>(3, 3)
9
2<jupyter_text>**6-Operations to Elements**You want to apply some function to multiple elements in an array.Numpy’s vectorize class converts a function into a function that can apply to multiple elements in an array or slice of an array.<jupyter_code>#Load Library
import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix)

#Create a function that adds 100 to something
add_100 =lambda i: i+100

#Convert it into a vectorized function
vectorized_add_100= np.vectorize(add_100)

#Apply function to all elements in matrix
print(vectorized_add_100(matrix))<jupyter_output>[[1 2 3]
 [4 5 6]
 [7 8 9]]
[[101 102 103]
 [104 105 106]
 [107 108 109]]<jupyter_text>**7-Max & Min values**We use Numpy’s max and min functions<jupyter_code>#Load Library
import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix)

#Return the max element
print(np.max(matrix))

#Return the min element
print(np.min(matrix))

#To find the max element in each column
print(np.max(matrix,axis=0))

#To find the max element in each row
print(np.max(matrix,axis=1))<jupyter_output>[[1 2 3]
 [4 5 6]
 [7 8 9]]
9
1
[7 8 9]
[3 6 9]<jupyter_text>**8-Calc Avg, Var & Std Dev**When you want to calculate some descriptive statistics about an array<jupyter_code>#Load Library
import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix)

#Mean
print(np.mean(matrix))

#Standard Dev.
print(np.std(matrix))

#Variance
print(np.var(matrix))<jupyter_output>[[1 2 3]
 [4 5 6]
 [7 8 9]]
5.0
2.581988897471611
6.666666666666667<jupyter_text>**9-Reshaping Arrays**When you want to reshape an array(changing the number of rows and columns) without changing the elements.<jupyter_code>#Load Library
import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix)

#Reshape
print(matrix.reshape(9,1))

#Here -1 says as many columns as needed and 1 row
print(matrix.reshape(1,-1))

#If we provide only 1 value Reshape would return a 1-d array of that length
print(matrix.reshape(9))

#We can also use the Flatten method to convert a matrix to 1-d array
print(matrix.flatten())<jupyter_output>[[1 2 3]
 [4 5 6]
 [7 8 9]]
[[1]
 [2]
 [3]
 [4]
 [5]
 [6]
 [7]
 [8]
 [9]]
[[1 2 3 4 5 6 7 8 9]]
[1 2 3 4 5 6 7 8 9]
[1 2 3 4 5 6 7 8 9]<jupyter_text>**10-Transposing**By transposing you interchange the rows and columns of a Matrix<jupyter_code>#Load Library
import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix)

#Transpose the matrix
print(matrix.T)<jupyter_output>[[1 2 3]
 [4 5 6]
 [7 8 9]]
[[1 4 7]
 [2 5 8]
 [3 6 9]]<jupyter_text>**11-Determinant & Rank**The rank of a Matrix is the number of dimensions of the vector space spanned by its rows or columns<jupyter_code>#Load Library
import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix)

#Calculate the Determinant
print(np.linalg.det(matrix))

#Calculate the Rank
print(np.linalg.matrix_rank(matrix))<jupyter_output>[[1 2 3]
 [4 5 6]
 [7 8 9]]
0.0
2<jupyter_text>**12-Diagonal**When you need to extract only the diagonal elements of a matrix<jupyter_code>#Load Library
import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix)

#Print the Principal diagonal
print(matrix.diagonal())

#Print the diagonal one above the Principal diagonal
print(matrix.diagonal(offset=1))

#Print the diagonal one below Principal diagonal
print(matrix.diagonal(offset=-1))<jupyter_output>[[1 2 3]
 [4 5 6]
 [7 8 9]]
[1 5 9]
[2 6]
[4 8]<jupyter_text>**13-Trace**Trace of a Matrix is the sum of elements on the Principal Diagonal of the Matrix.<jupyter_code>#Load Library
import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix)

#Print the Trace
print(matrix.trace())<jupyter_output>[[1 2 3]
 [4 5 6]
 [7 8 9]]
15<jupyter_text>**14-Eigenvalues & Eigenvectors**Eigenvectors are widely used in Machine Learning libraries. Intutively given a linear transformation represented by a matrix,A, eigenvectors are vectors that when that transformation is applied, change only in scale(not direction).More formallyAv=Kv<jupyter_code>#Load Library
import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix)

# Calculate the Eigenvalues and Eigenvectors of that Matrix
eigenvalues ,eigenvectors=np.linalg.eig(matrix)
print(eigenvalues)
print(eigenvectors)<jupyter_output>[[1 2 3]
 [4 5 6]
 [7 8 9]]
[ 1.61168440e+01 -1.11684397e+00 -1.30367773e-15]
[[-0.23197069 -0.78583024  0.40824829]
 [-0.52532209 -0.08675134 -0.81649658]
 [-0.8186735   0.61232756  0.40824829]]<jupyter_text>**15-Dot & Cross Products**<jupyter_code>#Load Library
import numpy as np

#Create vector-1
vector_1 = np.array([ 1,2,3 ])
#Create vector-2
vector_2 = np.array([ 4,5,6 ])

#Calculate Dot Product
print("Dot Product:",np.dot(vector_1,vector_2))

#Alternatively you can use @ to calculate dot products
print("Dot Product using @:",vector_1 @ vector_2)

# Cross operator
print("Cross Product:",np.cross(vector_1,vector_2))<jupyter_output>Dot Product: 32
Dot Product using @: 32
Cross Product: [-3  6 -3]<jupyter_text>**16-Add, Subtract, Multiply, Sort, nUnique**Numpy overloads all basic functions so that they can operate on arrays for example the plus operator +, multiply operator * etc. are all overloaded for arrays. Special functions such as square root and log are also available. These operations occur element-wise, thus when interacting with multiple arrays, they must all have same shape. There are many other operations possible such as exp, sin, cos, tan, and abs. We can also sort(), find min() and max(), determine unique() values, diagonal() elements of a matrix and sort() elements. If we add arg in front of min or max, we can also get the index of the minimum or maximum value in the array.<jupyter_code>#Load Library
import numpy as np

#Create Matrix
matrix_1 = np.array([[1,2,3],[4,5,6],[7,8,9]])
matrix_2 = np.array([[7,8,9],[4,5,6],[1,2,3]])

#Add the 2 Matrices
print(np.add(matrix_1,matrix_2))

#Subtraction
print(np.subtract(matrix_1,matrix_2))

#Multiplication(Element wise, not Dot Product)
print(matrix_1*matrix_2)

# Get unqiue values
print(f"Unique values: {np.unique(matrix_1)}")

# Sort values in the multidimensional array
print(f"Sorted:{np.sort(matrix_1)}")<jupyter_output>[[ 8 10 12]
 [ 8 10 12]
 [ 8 10 12]]
[[-6 -6 -6]
 [ 0  0  0]
 [ 6  6  6]]
[[ 7 16 27]
 [16 25 36]
 [ 7 16 27]]
Unique values: [1 2 3 4 5 6 7 8 9]
Sorted:[[1 2 3]
 [4 5 6]
 [7 8 9]]<jupyter_text>**ArgPartition, Allclose, Clip, Extract, Where, Percentile*** NumPy has this amazing function which can find N largest values index. The output will be the N largest values index, and then we can sort the values if needed.* Allclose() is used for matching two arrays and getting the output in terms of a boolean value. It will return False if items in two arrays are not equal within a tolerance. It is a great way to check if two arrays are similar, which can actually be difficult to implement manually.* Clip() is used to keep values in an array within an interval. Sometimes, we need to keep the values within an upper and lower limit. For the mentioned purpose, we can make use of NumPy’s clip(). Given an interval, values outside the interval are clipped to the interval edges.* Extract() as the name goes, is used to extract specific elements from an array based on a certain condition. With extract(), we can also use conditions like and and or.* Where() is used to return elements from an array that satisfy a certain condition. It returns the index position of values that fall in a certain condition. This is almost similar to the where condition that we use in SQL* Percentile() is used to compute the nth percentile of the array elements along the specified axis.<jupyter_code># Argpartition 
x = np.array([12, 10, 12, 0, 6, 8, 9, 1, 16, 4, 6, 0])
index_val = np.argpartition(x, -4)[-4:]

np.sort(x[index_val])
# AllClose
array1 = np.array([0.12,0.17,0.24,0.29])
array2 = np.array([0.13,0.19,0.26,0.31])

# with a tolerance of 0.1, it should return False:
print(np.allclose(array1,array2,0.1))

# with a tolerance of 0.2, it should return True:
print(np.allclose(array1,array2,0.2))
# Clip
x = np.array([3, 17, 14, 23, 2, 2, 6, 8, 1, 2, 16, 0])
np.clip(x,2,5)
# Apply condition on extract directly
array = np.random.randint(20, size=12)
np.extract(((array < 3) | (array > 15)), array)
# Where
y = np.array([1,5,6,8,1,7,3,6,9])

# First will replace the values that match the condition, 
# second will replace the values that does not
np.where(y>5, "Hit", "Miss")
# Percentile

#50th Percentile of a, axis = 0 :  6.0
a = np.array([1,5,6,8,1,7,3,6,9])
print("50th Percentile of a, axis = 0 : ",np.percentile(a, 50, axis =0))

#30th Percentile of b, axis = 0
b = np.array([[10, 7, 4], [3, 2, 1]])
print("30th Percentile of b, axis = 0 : ", np.percentile(b, 30, axis =0))<jupyter_output>50th Percentile of a, axis = 0 :  6.0
30th Percentile of b, axis = 0 :  [5.1 3.5 1.9]<jupyter_text>**18-Inverse**This is used when you want to calculate the inverse of a Square Matrix<jupyter_code>#Load Library
import numpy as np

#Create a Matrix
matrix = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(matrix)

#Calculate its inverse
print(np.linalg.inv(matrix))<jupyter_output>[[1 2 3]
 [4 5 6]
 [7 8 9]]<jupyter_text>**19-Random values**It can sometimes be useful to return the same random numbers to get predictable, repeatable results. We can do so by setting the ‘Seed’ (An Integer) of the pseudorandom generator. Random processes with the same seed would always produce the same result.np.random enables us to define the random arrays. The function rand() defines a random array if the argument includes a single integer e.g. 5 in the example above and a random matrix if the argument is a combination of integers e.g. (5,4) in the matrix example above. This function returns value within the range of 0 to 1 but if we want integers we should instead use randint(). It takes the first argument as start value, second argument as end value and the third argument as the shape of the array.<jupyter_code>print(np.random.rand(5))
print(np.random.rand(5,4))
#Creates a random normal distribution
np.random.randn(4,2)
#Selects 10 numbers between 0 & 100
np.random.randint(0,100,10) 
#Gives 50 equally spaced points between 0 & 10
arr1=np.linspace(0,10,50)
arr1<jupyter_output><empty_output><jupyter_text>**20-Concatenate**<jupyter_code># Concatenation of multi-dimensional arrays
arr1 = np.array([[1,2], [3,4]], int)
arr2 = np.array([[5,6], [7,8]], int)
print(np.concatenate((arr1, arr2)))

# Based Row
print(np.concatenate((arr1, arr2), axis=0))

# Based on Column
print(np.concatenate((arr1, arr2), axis=1))<jupyter_output>[[1 2]
 [3 4]
 [5 6]
 [7 8]]
[[1 2]
 [3 4]
 [5 6]
 [7 8]]
[[1 2 5 6]
 [3 4 7 8]]<jupyter_text>**21-BroadCasting**The term broadcasting refers to the ability of NumPy to treat arrays of different shapes during arithmetic operations. Arithmetic operations on arrays are usually done on corresponding elements. If two arrays are of exactly the same shape, then these operations are smoothly performed.<jupyter_code>#Broadcasting
arr[3:]=100
arr<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12004001333777926, 'fraction_numerical': 0.06302100700233411, 'mean_word_length': 5.100895036615134, 'pattern_counts': {'":': 0, '<': 73, '<?xml version=': 0, '>': 74, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '661240', 'license': 'MIT', 'n_tokens_mistral': 5696, 'n_tokens_neox': 5157, 'n_words': 1981, 'path': 'ML0.1-Numpy.ipynb', 'repo_name': 'KumarPython/Machine-Learning', 'size': '14995'}