<jupyter_start><jupyter_text>Numpy in python  What is a NumPy in Python?NumPy is a Python library used for working with arrays.It also has functions for working in domain of linear algebra, fourier transform, and matrices.NumPy was created in 2005 by <NAME>. It is an open source project and you can use it freely.NumPy stands for Numerical Python.  Difference between Numpy and Pandas ![aa.jpeg](attachment:aa.jpeg)  Why Use NumPy?In Python we have lists that serve the purpose of arrays, but they are slow to process.NumPy aims to provide an array object that is up to 50x faster than traditional Python lists.The array object in NumPy is called ndarray, it provides a lot of supporting functions that make working with ndarray very easy.Arrays are very frequently used in data science, where speed and resources are very important.  NumPy Getting StartedInstallation of NumPyIf you have Python and PIP already installed on a system, then installation of NumPy is very easy.Install it using this command:<jupyter_code>pip install numpy<jupyter_output>Collecting numpy
  Downloading numpy-1.20.2-cp38-cp38-macosx_10_9_x86_64.whl (16.0 MB)
[K     |████████████████████████████████| 16.0 MB 7.0 MB/s eta 0:00:01
[?25hInstalling collected packages: numpy
Successfully installed numpy-1.20.2
Note: you may need to restart the kernel to use updated packages.<jupyter_text>Import NumPyOnce NumPy is installed, import it in your applications by adding the import keyword:<jupyter_code>import numpy
import numpy 

arr = numpy.array([1, 2, 3, 4, 5]) 

print(arr)<jupyter_output>[1 2 3 4 5]<jupyter_text>NumPy as npNumPy is usually imported under the np alias.alias: In Python alias are an alternate name for referring to the same thing.Create an alias with the as keyword while importing<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>Now the NumPy package can be referred to as np instead of numpy<jupyter_code>arr = np.array([1, 2, 3, 4, 5]) 

print(arr)<jupyter_output>[1 2 3 4 5]<jupyter_text>Checking NumPy VersionThe version string is stored under version attribute.<jupyter_code>print(np.__version__)<jupyter_output>1.20.2<jupyter_text>NumPy Creating ArraysCreate a NumPy ndarray ObjectNumPy is used to work with arrays. The array object in NumPy is called ndarray.We can create a NumPy ndarray object by using the array() function.<jupyter_code>arr = np.array([1, 2, 3, 4, 5])

print(arr)

print(type(arr))<jupyter_output>[1 2 3 4 5]
<class 'numpy.ndarray'><jupyter_text>Dimensions in ArraysA dimension in arrays is one level of array depth (nested arrays).__0-D Arrays__ 0-D arrays, or Scalars, are the elements in an array. Each value in an array is a 0-D array__1-D Arrays__ An array that has 0-D arrays as its elements is called uni-dimensional or 1-D array.These are the most common and basic arrays.__2-D Arrays__ An array that has 1-D arrays as its elements is called a 2-D array.These are often used to represent matrix or 2nd order tensors__3-D arrays__ An array that has 2-D arrays (matrices) as its elements is called 3-D array.These are often used to represent a 3rd order tensorCheck Number of Dimensions? NumPy Arrays provides the ndim attribute that returns an integer that tells us how many dimensions the array have.<jupyter_code>a = np.array(42)
b = np.array([1, 2, 3, 4, 5])
c = np.array([[1, 2, 3], [4, 5, 6]])
d = np.array([[[1, 2, 3], [4, 5, 6]], [[1, 2, 3], [4, 5, 6]]])

print(a.ndim) 
print(b.ndim) 
print(c.ndim) 
print(d.ndim)<jupyter_output>0
1
2
3<jupyter_text>NumPy Array Indexing Access Array ElementsArray indexing is the same as accessing an array element.You can access an array element by referring to its index number.The indexes in NumPy arrays start with 0, meaning that the first element has index 0, and the second has index 1 etc.<jupyter_code>#Get the first element from the following array:
arr = np.array([1, 2, 3, 4])

print(arr[0])
#Get the second element from the following array.
arr = np.array([1, 2, 3, 4])

print(arr[1])<jupyter_output>2<jupyter_text>Access 2-D ArraysTo access elements from 2-D arrays we can use comma separated integers representing the dimension and the index of the element.<jupyter_code>#Access the 2nd element on 1st dim:

arr = np.array([[1,2,3,4,5], [6,7,8,9,10]])

print('2nd element on 1st dim: ', arr[0, 1])<jupyter_output>2nd element on 1st dim:  2<jupyter_text>Access 3-D ArraysTo access elements from 3-D arrays we can use comma separated integers representing the dimensions and the index of the element.<jupyter_code>#Access the third element of the second array of the first array:
arr = np.array([[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]])

print(arr[0, 1, 2])<jupyter_output>6<jupyter_text>Negative IndexingUse negative indexing to access an array from the end<jupyter_code>arr = np.array([[1,2,3,4,5], [6,7,8,9,10]])

print('Last element from 2nd dim: ', arr[1, -1])<jupyter_output>Last element from 2nd dim:  10<jupyter_text>NumPy Array SlicingSlicing arraysSlicing in python means taking elements from one given index to another given index.We pass slice instead of index like this: [start:end].We can also define the step, like this: [start:end:step].If we don't pass start its considered 0If we don't pass end its considered length of array in that dimensionIf we don't pass step its considered 1<jupyter_code>#Slice elements from index 1 to index 5 from the following array:
    
arr = np.array([1, 2, 3, 4, 5, 6, 7])

print(arr[1:5])<jupyter_output>[2 3 4 5]<jupyter_text>Note: The result includes the start index, but excludes the end index<jupyter_code>#Slice elements from index 4 to the end of the array:
arr = np.array([1, 2, 3, 4, 5, 6, 7])

print(arr[4:])<jupyter_output>[5 6 7]<jupyter_text>Negative Slicing Use the minus operator to refer to an index from the end:<jupyter_code>#Slice from the index 3 from the end to index 1 from the end:
arr = np.array([1, 2, 3, 4, 5, 6, 7])

print(arr[-3:-1])<jupyter_output>[5 6]<jupyter_text>STEP Use the step value to determine the step of the slicing<jupyter_code>#Return every other element from index 1 to index 5

arr = np.array([1, 2, 3, 4, 5, 6, 7])

print(arr[1:5:2])<jupyter_output><empty_output><jupyter_text>Slicing 2-D Arrays<jupyter_code>#From the second element, slice elements from index 1 to index 4 (not included):
arr = np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]])

print(arr[1, 1:4])
#From both elements, return index 2:

arr = np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]])

print(arr[0:2, 2])<jupyter_output>[3 8]<jupyter_text>Data Types in PythonBy default Python have these data types:strings - used to represent text data, the text is given under quote marks. e.g. "ABCD" integer - used to represent integer numbers. e.g. -1, -2, -3 float - used to represent real numbers. e.g. 1.2, 42.42 boolean - used to represent True or False. complex - used to represent complex numbers. e.g. 1.0 + 2.0j, 1.5 + 2.5jData Types in NumPy NumPy has some extra data types, and refer to data types with one character, like i for integers, u for unsigned integers etc.Below is a list of all data types in NumPy and the characters used to represent them.- i - integer- b - boolean - u - unsigned integer- f - float- c - complex float - m - timedelta- M - datetime- O - object- S - string- U - unicode string- V - fixed chunk of memory for other type ( void )  Checking the Data Type of an ArrayThe NumPy array object has a property called dtype that returns the data type of the array:<jupyter_code>arr = np.array([1, 2, 3, 4])

print(arr.dtype)<jupyter_output>int64<jupyter_text>Creating Arrays With a Defined Data TypeWe use the array() function to create arrays, this function can take an optional argument: dtype that allows us to define the expected data type of the array elements:<jupyter_code>arr = np.array([1, 2, 3, 4], dtype='S')

print(arr)
print(arr.dtype)
#Create an array with data type 4 bytes integer: 
arr = np.array([1, 2, 3, 4], dtype='i4')

print(arr)
print(arr.dtype)<jupyter_output>[1 2 3 4]
int32<jupyter_text>Converting Data Type on Existing ArraysThe best way to change the data type of an existing array, is to make a copy of the array with the astype() method.The astype() function creates a copy of the array, and allows you to specify the data type as a parameter.The data type can be specified using a string, like 'f' for float, 'i' for integer etc. or you can use the data type directly like float for float and int for integer.<jupyter_code>#Change data type from float to integer by using 'i' as parameter value:

arr = np.array([1.1, 2.1, 3.1])

newarr = arr.astype('i')

print(newarr)
print(newarr.dtype)<jupyter_output>[1 2 3]
int32<jupyter_text>NumPy Array Copy vs View The Difference Between Copy and ViewThe main difference between a copy and a view of an array is that the copy is a new array, and the view is just a view of the original array.The copy owns the data and any changes made to the copy will not affect original array, and any changes made to the original array will not affect the copy.The view does not own the data and any changes made to the view will affect the original array, and any changes made to the original array will affect the view.<jupyter_code>#COPY

arr = np.array([1, 2, 3, 4, 5])
x = arr.copy()
arr[0] = 42

print(arr) 
print(x)
#View
arr = np.array([1, 2, 3, 4, 5])
x = arr.view()
arr[0] = 42

print(arr) 
print(x)<jupyter_output>[42  2  3  4  5]
[42  2  3  4  5]<jupyter_text>Check if Array Owns it's DataAs mentioned above, copies owns the data, and views does not own the data, but how can we check this?Every NumPy array has the attribute base that returns None if the array owns the data.Otherwise, the base attribute refers to the original object.<jupyter_code>arr = np.array([1, 2, 3, 4, 5])

x = arr.copy()
y = arr.view()

print(x.base)
print(y.base)<jupyter_output>None
[1 2 3 4 5]<jupyter_text>The copy returns None.The view returns the original array.  Get the Shape of an ArrayNumPy arrays have an attribute called shape that returns a tuple with each index having the number of corresponding elements.<jupyter_code>arr = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])

print(arr.shape)<jupyter_output>(2, 4)<jupyter_text>Reshaping arraysReshaping means changing the shape of an array.The shape of an array is the number of elements in each dimension.By reshaping we can add or remove dimensions or change number of elements in each dimension.Reshape From 1-D to 2-DConvert the following 1-D array with 12 elements into a 2-D array.The outermost dimension will have 4 arrays, each with 3 elements:<jupyter_code>arr = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])

newarr = arr.reshape(4, 3)

print(newarr)<jupyter_output>[[ 1  2  3]
 [ 4  5  6]
 [ 7  8  9]
 [10 11 12]]<jupyter_text>Flattening the arraysFlattening array means converting a multidimensional array into a 1D array.We can use reshape(-1) to do this.<jupyter_code>arr = np.array([[1, 2, 3], [4, 5, 6]])

newarr = arr.reshape(-1)

print(newarr)<jupyter_output>[1 2 3 4 5 6]<jupyter_text>NumPy Array IteratingIterating ArraysIterating means going through elements one by one.As we deal with multi-dimensional arrays in numpy, we can do this using basic for loop of python.If we iterate on a 1-D array it will go through each element one by one.<jupyter_code>#Iterate on the elements of the following 1-D array:
arr = np.array([1, 2, 3])

for x in arr:
  print(x)
#Iterate on the elements of the following 2-D array:

arr = np.array([[1, 2, 3], [4, 5, 6]])

for x in arr:
  print(x)<jupyter_output>[1 2 3]
[4 5 6]<jupyter_text>Iterating Array With Different Data TypesWe can use op_dtypes argument and pass it the expected datatype to change the datatype of elements while iterating.NumPy does not change the data type of the element in-place (where the element is in array) so it needs some other space to perform this action, that extra space is called buffer, and in order to enable it in nditer() we pass flags=['buffered'].<jupyter_code>arr = np.array([1, 2, 3])

for x in np.nditer(arr, flags=['buffered'], op_dtypes=['S']):
  print(x)<jupyter_output>b'1'
b'2'
b'3'<jupyter_text>Iterating With Different Step SizeWe can use filtering and followed by iteration.<jupyter_code># Iterate through every scalar element of the 2D array skipping 1 element:

arr = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])

for x in np.nditer(arr[:, ::2]):
  print(x)<jupyter_output>1
3
5
7<jupyter_text>Enumerated Iteration Using ndenumerate()Enumeration means mentioning sequence number of somethings one by one.Sometimes we require corresponding index of the element while iterating, the ndenumerate() method can be used for those usecases.<jupyter_code># Enumerate on following 1D arrays elements:
arr = np.array([1, 2, 3])

for idx, x in np.ndenumerate(arr):
  print(idx, x)
# Enumerate on following 2D array's elements:

arr = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])

for idx, x in np.ndenumerate(arr):
  print(idx, x)<jupyter_output>(0, 0) 1
(0, 1) 2
(0, 2) 3
(0, 3) 4
(1, 0) 5
(1, 1) 6
(1, 2) 7
(1, 3) 8<jupyter_text>NumPy Joining Array  Joining NumPy ArraysJoining means putting contents of two or more arrays in a single array.In SQL we join tables based on a key, whereas in NumPy we join arrays by axes.We pass a sequence of arrays that we want to join to the concatenate() function, along with the axis. If axis is not explicitly passed, it is taken as 0.<jupyter_code>#Join two arrays

arr1 = np.array([1, 2, 3])

arr2 = np.array([4, 5, 6])

arr = np.concatenate((arr1, arr2))

print(arr)
# Join two 2-D arrays along rows (axis=1)
arr1 = np.array([[1, 2], [3, 4]])

arr2 = np.array([[5, 6], [7, 8]])

arr = np.concatenate((arr1, arr2), axis=1)

print(arr)<jupyter_output>[[1 2 5 6]
 [3 4 7 8]]<jupyter_text>NumPy Splitting Array  Splitting NumPy ArraysSplitting is reverse operation of Joining.Joining merges multiple arrays into one and Splitting breaks one array into multiple.We use array_split() for splitting arrays, we pass it the array we want to split and the number of splits.<jupyter_code># Split the array in 3 parts:

arr = np.array([1, 2, 3, 4, 5, 6])

newarr = np.array_split(arr, 3)

print(newarr)<jupyter_output>[array([1, 2]), array([3, 4]), array([5, 6])]<jupyter_text>Split Into ArraysThe return value of the array_split() method is an array containing each of the split as an array.If you split an array into 3 arrays, you can access them from the result just like any array element:<jupyter_code>arr = np.array([1, 2, 3, 4, 5, 6])

newarr = np.array_split(arr, 3)

print(newarr[0])
print(newarr[1])
print(newarr[2])<jupyter_output>[1 2]
[3 4]
[5 6]<jupyter_text>Splitting 2-D ArraysUse the same syntax when splitting 2-D arrays.Use the array_split() method, pass in the array you want to split and the number of splits you want to do.<jupyter_code># Split the 2-D array into three 2-D arrays.

arr = np.array([[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12]])

newarr = np.array_split(arr, 3)

print(newarr)<jupyter_output>[array([[1, 2],
       [3, 4]]), array([[5, 6],
       [7, 8]]), array([[ 9, 10],
       [11, 12]])]<jupyter_text>NumPy Searching Arrays Searching ArraysYou can search an array for a certain value, and return the indexes that get a match.To search an array, use the where() method.<jupyter_code># Find the indexes where the value is 4:

arr = np.array([1, 2, 3, 4, 5, 4, 4])

x = np.where(arr == 4)

print(x)<jupyter_output>(array([3, 5, 6]),)<jupyter_text>Search SortedThere is a method called searchsorted() which performs a binary search in the array, and returns the index where the specified value would be inserted to maintain the search order.<jupyter_code># Find the indexes where the value 7 should be inserted:

arr = np.array([6, 7, 8, 9])

x = np.searchsorted(arr, 7)

print(x)<jupyter_output>1<jupyter_text>NumPy Sorting Arrays Sorting ArraysSorting means putting elements in an ordered sequence.Ordered sequence is any sequence that has an order corresponding to elements, like numeric or alphabetical, ascending or descending.The NumPy ndarray object has a function called sort(), that will sort a specified array.<jupyter_code>arr = np.array([3, 2, 0, 1])

print(np.sort(arr))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1026005312249058, 'fraction_numerical': 0.03897708320464513, 'mean_word_length': 4.74724884629038, 'pattern_counts': {'":': 0, '<': 111, '<?xml version=': 0, '>': 111, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '35', 'dir': 'jupyter-structured-clean-dedup', 'id': '56948', 'license': 'Apache-2.0', 'n_tokens_mistral': 5845, 'n_tokens_neox': 5347, 'n_words': 2318, 'path': 'lesson03/lecture/numpy/numpy.ipynb', 'repo_name': 'benesom/redi-da-cph-fall21', 'size': '16198'}