<jupyter_start><jupyter_text>Python: The numpy library **Goal**: manipulate matrices or multidimensional arrays with the numpy package!  Introduction to numpy Numpy is one of the fundamental package for scientific computing in Python. The numpy library (http://www.numpy.org/) allows you to perform numerical calculations with Python. It introduces an easy management of arrays of numbers. To use numpy, you must first import the numpy package with the following instruction ``` import numpy ```.<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>Arrays with numpy In this section, we will discover the numpy tables. So, the main data structure in numpy is the ``` ndarray ```. Arrays can be created with ``` numpy.array() ```. Square brackets are used to delimit lists of elements in arrays. In summary, a vector represents a list and a matrix represents a list of lists.<jupyter_code>vector = np.array([1, 2, 3, 4, 5])
print(vector)
matrix = np.array([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0], [7.0, 8.0, 9.0]])
print(matrix)<jupyter_output>[[1. 2. 3.]
 [4. 5. 6.]
 [7. 8. 9.]]<jupyter_text>Size of an array When manipulating matrices (matrix multiplication for example), it is very useful to know the ``` size of the array ```. Otherwise, you may get errors. So to do this, we have the array property ``` ndarray.shape ```. For vectors, shape will return a tuple with an element. On the other hand, the size of a matrix has two elements, the number of rows and the number of columns.<jupyter_code>vector.shape
matrix.shape<jupyter_output><empty_output><jupyter_text>Read dataset using numpy To read a dataset directly with the numpy library, we use the numpy ``` genfromtxt() ``` function.<jupyter_code>world_alcohol = np.genfromtxt("world_alcohol.csv", delimiter=",")
world_alcohol
print(type(world_alcohol))<jupyter_output><class 'numpy.ndarray'><jupyter_text>We can clearly see that there is a lot of nan in the result. In reality, they are not nan, we will see later how to correct this problem. In fact, when we talk about matrices, we must have only numbers in the dataset. However numpy can accept many types of data, not only numbers. But the particularity of a numpy array is that all the values of the different columns must have the same data type by default. The default format type used by the genformtext function is float, see the documentation: https://numpy.org/doc/stable/reference/generated/numpy.genfromtxt.html.  Data types in numpy Here we will see the main data types in numpy which have slight differences with the Python data types. We will also see the particularity with numpy arrays. The data types accepted by numpy are: ```bool```, ```integer```, ```float``` and ```string```. To know the data type of an array, we use the ```dtype``` method of the numpy ```ndarray object```.<jupyter_code>vector.dtype
matrix.dtype
world_alcohol.dtype<jupyter_output><empty_output><jupyter_text>Display data correctly We have seen previously that our dataset displays nan on some columns. In this section, we will see how to display the data correctly. As a reminder, ```nan``` means ```not a number```, it is a ```data type``` that is used to say that the ```value is missing```. There is another type of data that is ```na (not available)``` for items that are not available. So you have to know that ```nan``` and ```na``` are data types.<jupyter_code>world_alcohol<jupyter_output><empty_output><jupyter_text>To display the correct values, we use the ```dtype=U75``` parameter of the ```genfromtxt``` function. This is to force numpy to read each value in the ```U75 (Unicode 75 bit)``` format.<jupyter_code>world_alcohol = np.genfromtxt("world_alcohol.csv", delimiter=",", dtype="U75")
world_alcohol<jupyter_output><empty_output><jupyter_text>We notice that our dataset has a header. To remove it, we use the ```skip_header``` parameter which we will set to a ```1``` to skip the first row.<jupyter_code>world_alcohol = np.genfromtxt("world_alcohol.csv", delimiter=",", dtype="U75", skip_header=1)
world_alcohol<jupyter_output><empty_output><jupyter_text>Extract a value from a numpy array The extraction of an element from a numpy array (vectors) is done in a similar way to a list. For matrices, indexing is similar as for lists of list.<jupyter_code># example for vector
vector, vector[0]
# example for matrice
matrix, matrix[0,0]<jupyter_output><empty_output><jupyter_text>Training In this section, we will try to answer the following questions:* assign the number of liters of wine drunk by an Ivorian in 1985 to the variable value_ivory_1985, this corresponds to the data in the 3rd line* assign the name of the country in the 2nd line to the variable second_country_name<jupyter_code>world_alcohol
# number of liters of wine drunk by an Ivorian in 1985
value_ivory_1985 = world_alcohol[2,4]
print(value_ivory_1985)
# name of the country in the 2nd line
second_country_name = world_alcohol[1,2]
print(second_country_name)<jupyter_output>Uruguay<jupyter_text>Extract a vector of values from a numpy array The extraction of a vector from a numpy array is done in the same way as the extraction of an element. It consists in a kind of **slicing** from a list. However, the extraction of a matrix is a bit more complex. You can extract a row or column vector, or in a more advanced way, extract a subset of a matrix.<jupyter_code># example for vector
vector
sub_vector = vector[0:3]
sub_vector
# example for matrix
matrix
# column extraction
first_column = matrix[:,0]
first_column
second_column = matrix[:,1]
second_column
third_column = matrix[:,2]
third_column
# row extraction
first_row = matrix[0,:]
first_row
second_row = matrix[1,:]
second_row
third_row = matrix[2,:]
third_row<jupyter_output><empty_output><jupyter_text>Training In this section, we will try to answer the following questions:* assign all the 3rd column of world_alcohol to the countries variable* assign the whole 5th column of world_alcohol to the variable alcohol_consumption<jupyter_code>countries = world_alcohol[:,2]
countries
alcohol_consumption = world_alcohol[:,4]
alcohol_consumption<jupyter_output><empty_output><jupyter_text>Extract an array of values from a numpy array In this section we will see how to extract an array of values (matrix or sub-matrix) from a numpy array of values (matrix). Extraction is done in the same way just by double slicing the rows and columns of a matrix.<jupyter_code># example
matrix
first_sub_matrix = matrix[:2,:2]
first_sub_matrix
second_sub_matrix = matrix[:2,1:3]
second_sub_matrix
third_sub_matrix = matrix[1:3,:2]
third_sub_matrix
fourth_sub_matrix = matrix[1:3,1:3]
fourth_sub_matrix<jupyter_output><empty_output><jupyter_text>These extractions are interesting when the determinant of a matrix has to be calculated.  Training In this section, we will try to answer the following questions:* assign all lines of the first 2 columns of world_acohol to the variable first_two_columns* assign the first 10 rows of the first column of world_alcohol to the variable first_ten_years* assign the first 10 rows of all world_alcohol columns to the variable first_ten_rows* assign the first 20 rows of the world_alcohol index 1 and 2 columns to the variable first_twenty_regions<jupyter_code>first_two_colomns = world_alcohol[:,:2]
first_two_colomns
first_ten_years = world_alcohol[:10,0]
first_ten_years
first_ten_rows = world_alcohol[:10,:]
first_ten_rows
first_twenty_regions = world_alcohol[:20,1:3]
first_twenty_regions<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08388814913448735, 'fraction_numerical': 0.015579227696404794, 'mean_word_length': 5.840619307832423, 'pattern_counts': {'":': 0, '<': 55, '<?xml version=': 0, '>': 55, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '39884', 'license': 'MIT', 'n_tokens_mistral': 2258, 'n_tokens_neox': 2200, 'n_words': 959, 'path': 'courses/1. The numpy library in Python.ipynb', 'repo_name': 'Niangmohamed/Introduction-to-Data-Science-With-Python', 'size': '7510'}