<jupyter_start><jupyter_text>Lab 1: Introduction to NumPyMachine Learning 2020/2021 Author: Machine Learning Teaching Team TU Delft **WHAT** This nonmandatory lab consists of several programming and insight exercises/questions. **WHY** The exercises are meant to prepare you for using Python and NumPy in this course. **HOW** Follow the exercises in the notebooks either on your own or with a fellow student. For questions and feedback please consult the TA's during the lab session.  We advise you to follow this notebook and use it as a reference for later. __Make sure that you have followed the Python tutorial (intro1-python).__ These tutorials cover a whole lot of Python and NumPy and you don't have to know everything by heart right away. But it is important to be aware of the total picture as this helps while making later assignments. You will find that you will be better able to troubleshoot if you run into a problem. If, after walking through these tutorials, you still feel uncomfortable with Python, we recommend the following tutorials:* [The Python Tutorial] * [Python Numpy Tutorial][Python Numpy Tutorial]: http://cs231n.github.io/python-numpy-tutorial/[The Python Tutorial]: https://docs.python.org/3/tutorial/index.html This tutorial consists of two steps: first, you will get familiarized with the basic operations on arrays in NumPy, after which you get to practice with a number of exercises. If you want more in-depth understanding of NumPy, we highly recommend [Stanfords NumPy tutorial]![Stanfords NumPy tutorial]: http://cs231n.github.io/python-numpy-tutorial/  Step 1: Array programming with NumPy In this step we will show you the basics of array manipulation in NumPy.We strongly advise you to reference and read the manuals and tutorials on the web to make sure you learn how to use modern array manipulation to unleash its true power. In machine learning we are dealing with massive amounts of data. This data is most often organised in tables (like for example in spreadsheets). When all data elements in a table are of the same datatype (like an integer or a floating point number) the table can be represented with a homogeneous array.Languages that are optimally suited for programming with data (like NumPy) are therefore equipped with array data types as an integral part of the language. An array is a data type to store lists of values. We can also create arrays of arrays, resulting in multi-dimensional arrays. NumPy provides tools to work with these multi-dimensional arrays in the form of the [`ndarray` class](https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.html).Let's first import the NumPy library:<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>Declaring a regular Python list<jupyter_code>list1 = [1, 2, 3, 4]
type(list1)<jupyter_output><empty_output><jupyter_text>Making a numpy array using Python lists<jupyter_code>array1 = np.array(list1)
array1
# The type of array1 is an ndarray: an n-dimensional array
type(array1)
print(array1)<jupyter_output>[1 2 3 4]<jupyter_text>We can also use more dimensions<jupyter_code># Declare an extra list.
list2 = [11, 22, 33, 44]
# Combine the lists into a 2-dimensional list.
lists = [list1, list2]
lists
# make a 2-dimensional NumPy array
array2 = np.array(lists)
array2<jupyter_output><empty_output><jupyter_text>And print their shapesWe would obviously expect a (4,1) and a (4,2)... or don't we?<jupyter_code>print("Arr1: ", array1.shape)
print("Arr2: ", array2.shape)<jupyter_output>Arr1:  (4,)
Arr2:  (2, 4)<jupyter_text>The best way to think about NumPy arrays is that they consist of two parts, a _data buffer_ which is just a block of raw elements, and a _view_ which describes how to interpret the data buffer.Here the shape `(4,)` means the array is indexed by a single index which runs from 0 to 3. The comma in `(4,)` is there to show that it is a tuple of integers, which is the data type of the shape attribute. Without the comma, it would simply be an integer, not a tuple of integers.In most situations the lack of a second dimension is not a problem. If it does turn into a problem (e.g. when you are trying to take a transpose of this vector) you can just call the `reshape` function on the array to generate a new view:<jupyter_code># Do note the double brackets, as the size is added as a tuple: (rows, columns)
array1 = array1.reshape((4,1))
array1.shape<jupyter_output><empty_output><jupyter_text>For the above examples, we happen to know what we stored in our array, but in some cases we are not aware (like when we imported lots of data). To find out, you can call `dtype`:<jupyter_code>array2.dtype<jupyter_output><empty_output><jupyter_text>Initializing regularly used arraysThere are also basic ways to initialize certain arrays which are used regularly, such as:<jupyter_code># The empty array (makes an array but doesn't do any initialisation)
print("Ex1: ", np.empty(5))

# Array of 5 floating point zeros
print("Ex2: ", np.zeros(5))

# Array of 5 floating point ones
print("Ex3: ", np.ones(5))

# Array of 5 integer incrementing numbers
print("Ex4: ", np.arange(5))

# Start at 5, stop at 20, do it in steps of 2
print("Ex5: ", np.arange(5, 20, 2))

# Making the identity matrix (ones on the diagonal)
print("Ex6: ")
print(np.eye(5))<jupyter_output>Ex1:  [0. 0. 0. 0. 0.]
Ex2:  [0. 0. 0. 0. 0.]
Ex3:  [1. 1. 1. 1. 1.]
Ex4:  [0 1 2 3 4]
Ex5:  [ 5  7  9 11 13 15 17 19]
Ex6: 
[[1. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0.]
 [0. 0. 1. 0. 0.]
 [0. 0. 0. 1. 0.]
 [0. 0. 0. 0. 1.]]<jupyter_text>Mathematical operations The power of NumPy lies in the mathematical operations you can apply on those arrays. These are applied element-wise, which means that the operation is performed on each individual element in the array. Linear Algebra operations, like matrix multiplication or dot product are performed with special NumPy functions, like `np.matmul` or `np.dot`.<jupyter_code>array3 = np.array([[1, 2, 3, 4], [8, 9, 10, 11]])
array3
# Element-wise multiplication
array3 * array3
# Element-wise subtraction
array3 - 5
# Division
1 / array3
# Raising to a power
array3 ** 3<jupyter_output><empty_output><jupyter_text>You can also apply functions to all elements in an array at once The nice thing about NumPy arrays is that it allows you to manipulate the data in arrays without writing explicit loops. For instance look at the addition of all elements in an array:<jupyter_code># Array of random numbers
a = np.random.rand(65536)
#calculate the sum of the elements in array
def loopsum(a):
    sum = 0
    for i in range(len(a)):
        sum += a[i]
    return sum
%timeit loopsum(a)
%timeit np.sum(a)<jupyter_output>14.6 ms ± 130 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
20.8 µs ± 164 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)<jupyter_text>You can see that the numpy loop is about 350 times faster than the explicit loop version.So be aware in this course to use built-in NumPy tools to manipulate and calculate with arrays.Some built-in functions of NumPy can be found [here].[here]: https://docs.scipy.org/doc/numpy/reference/ufuncs.htmlmath-operations  Indexing Arrays<jupyter_code>array4 = np.arange(0, 10)
array4<jupyter_output><empty_output><jupyter_text>There is a minor difference when it comes to indexing compared to Python lists: namely that a NumPy array allows an additional indexing method:<jupyter_code>list3 = [[1, 2, 3], [4, 5, 6]]
array5 = np.array(list3)

# Watch the brackets closely.
print("List: ", list3[1][2])
# Array can use two different approaches
print("Array ", array5[1, 2])
print("Array ", array5[1][2])<jupyter_output>List:  6
Array  6
Array  6<jupyter_text>Slicing arraysSometimes you do not want the full array, but just parts of it, we can use array slicing for this:<jupyter_code># Show original array
print(array5)
# We want the element from the second row and third column:
print(array5[1:2,2:3])
# We can also use it to set the value of multiple entries:
array4 = np.arange(0, 10)
array4[2:5] = 13
print(array4)<jupyter_output>[ 0  1 13 13 13  5  6  7  8  9]<jupyter_text>One important thing to note is that a slice is just another _view_ of the underlying data buffer. If you change data in the slice, you are actually changing the data in the underlying data buffer and thus in the orginal array. This is advantageous for the memory efficiency of your program, but sometimes it can cause errors when overlooked.<jupyter_code>array4 = np.arange(0, 10)
# Take a slice, consisting of the 3rd (index 2) to 6th (index 5) element.
slice_array4 = array4[2:6]
# We iterate over all values, setting them to 22
slice_array4[:] = 22
print(slice_array4)
print(array4)<jupyter_output>[22 22 22 22]
[ 0  1 22 22 22 22  6  7  8  9]<jupyter_text>To prevent this, we can also make a new copy. In this way we do not generate a view, but actually reserve new memory for the object we are making:<jupyter_code>array4 = np.arange(0, 10)
array5 = array4.copy()
print(array4)
print(array5)
array5[:] = 22
print("So did we make a copy?")
print(array4)
print(array5)
print("Seems we did.")<jupyter_output>[0 1 2 3 4 5 6 7 8 9]
[0 1 2 3 4 5 6 7 8 9]
So did we make a copy?
[0 1 2 3 4 5 6 7 8 9]
[22 22 22 22 22 22 22 22 22 22]
Seems we did.<jupyter_text>2D array slicing<jupyter_code>array6 = np.array([[2, 4, 6], [8, 10, 12], [14, 16, 18]])
print(array6)
# let's say you only want just the upper right square of 2x2 of the above matrix
# reminder: when indexing with array[start:stop]
# the element at start is *included*, while the elemnt at stop is *excluded*
array6[:2, 1:]<jupyter_output>[[ 2  4  6]
 [ 8 10 12]
 [14 16 18]]<jupyter_text>Fancy IndexingSometimes you don't want to retrieve every row, but perhaps skip a few entries. This is easily possible in Python. Let us assume we only want the 2nd, 3rd, 5th, and 7th row in the following example.<jupyter_code># Below we use a list comprehension (which you should have seen in Introduction to Programming as well)
# To generate an array with 10 rows, and each column goes from 0 to 10.
array7 = np.array([[j for i in range(10)] for j in range(10)])
print(array7)
# As we start at index 0, we actually want the following rows [1, 2, 4, 6].
# Also note the double brackets below.
print("With fancy indexing:")
print(array7[[1, 2, 4, 6]])<jupyter_output>[[0 0 0 0 0 0 0 0 0 0]
 [1 1 1 1 1 1 1 1 1 1]
 [2 2 2 2 2 2 2 2 2 2]
 [3 3 3 3 3 3 3 3 3 3]
 [4 4 4 4 4 4 4 4 4 4]
 [5 5 5 5 5 5 5 5 5 5]
 [6 6 6 6 6 6 6 6 6 6]
 [7 7 7 7 7 7 7 7 7 7]
 [8 8 8 8 8 8 8 8 8 8]
 [9 9 9 9 9 9 9 9 9 9]]
With fancy indexing:
[[1 1 1 1 1 1 1 1 1 1]
 [2 2 2 2 2 2 2 2 2 2]
 [4 4 4 4 4 4 4 4 4 4]
 [6 6 6 6 6 6 6 6 6 6]]<jupyter_text>You can do the above even in any order you wish.<jupyter_code>array_test = np.array([[0 for _ in range(3)] for _ in range(3)])
print(array_test)
array7[[6, 2, 4, 1]]<jupyter_output><empty_output><jupyter_text>Array Transposition<jupyter_code>array8 = np.arange(40).reshape((8, 5))
array8
# If you want to transpose a matrix you can do this in two ways:

print(np.transpose(array8))
# Or
print(array8.T)<jupyter_output>[[ 0  5 10 15 20 25 30 35]
 [ 1  6 11 16 21 26 31 36]
 [ 2  7 12 17 22 27 32 37]
 [ 3  8 13 18 23 28 33 38]
 [ 4  9 14 19 24 29 34 39]]
[[ 0  5 10 15 20 25 30 35]
 [ 1  6 11 16 21 26 31 36]
 [ 2  7 12 17 22 27 32 37]
 [ 3  8 13 18 23 28 33 38]
 [ 4  9 14 19 24 29 34 39]]<jupyter_text>Array ProcessingWe can also apply functions on arrays to retrieve specific information from them, or to process the information that is contained. In this section, we will survey a number of these handy functions.  Numpy WhereWhen you want to find the location of elements in an array, you can use `np.where`. This function takes in a condition and returns the indices of the array where the condition is true.<jupyter_code># A simple np.where example:
A = np.array([1, 2, 3, 4])
indices = np.where(A < 3)
print(indices)
print(A[indices])<jupyter_output>(array([0, 1], dtype=int64),)
[1 2]<jupyter_text>Numpy Any & AllAs shown before, we can apply a boolean operator on an array, which will return an array with boolean values:<jupyter_code>a = np.arange(9).reshape(3, 3)
bool_arr = a < 4
print(bool_arr)<jupyter_output>[[ True  True  True]
 [ True False False]
 [False False False]]<jupyter_text>Now, what if we want to return each row where any of the elements is true, we can do that with `any` in combination with `where`:<jupyter_code># Return the indices of the rows where any column (axis=1) is true
print((bool_arr.any(axis=1)))
indices = np.where(bool_arr.any(axis=1))
print(indices)
bool_arr[indices]
# You can see that the last row is excluded as this row doesn't contain any True values<jupyter_output>[ True  True False]
(array([0, 1], dtype=int64),)<jupyter_text>And what if we want all elements in the row to be true? We can use `all`:<jupyter_code># If all values are true, return true (else false)
indices = np.where(bool_arr.all(axis=1))
bool_arr[indices]
# You can see that only the first row is included as this row is the only one to contain only True values<jupyter_output><empty_output><jupyter_text>Numpy Unique and `in` checking<jupyter_code># Sometimes you just want to know all the unique values in a numpy array
# Luckily that function was already implemented for you
letters = ['A', 'B', 'C', 'D', 'D', 'A', 'E', 'F', 'G', 'H', 'Z']
np.unique(letters)
# We can also easily check whether a big array exists,
# if it exists within a 1D vector.
np.in1d(['X', 'C', 'M', 'Z'], letters)<jupyter_output><empty_output><jupyter_text>Step 2: Practice Numpy These are optional exercises, but highly recommended to get you familiar with some basic NumPy operations and tricks. If you get stuck, try to dive into the Numpy documentation first, to see if it can be of help as not all functions you need for the exercises are explained above.  Array Calculations and Array IndexingIn all exercises below you are not allowed to use a loop.<jupyter_code># Given two arrays A and B each of the same size calculate their sum (elementwise) and their product (elementwise).
A = np.arange(5)
B = np.arange(5, 10)

def sum_arrays(A, B):
    result = None
    # START ANSWER
    result = A + B
    # END ANSWER
    return result

def multiply_arrays(A, B):
    result = None
    # START ANSWER
    result = A * B
    # END ANSWER
    return result

sum_AB = sum_arrays(A, B)
assert (sum_AB == np.array([ 5,  7,  9, 11, 13])).all()
mult_AB = multiply_arrays(A, B)
assert (mult_AB == np.array([ 0,  6, 14, 24, 36])).all()

sum_AB, mult_AB
# Given an array A with shape (128,) calculate the mean of the elements at even indexes.
A = np.arange(128)

def mean_even_idx(A):
    result = 0
    # START ANSWER
    for i in A:
        if i % 2 == 0:
            result += i
    result = result/(A.shape[0]/2)
    # END ANSWER
    return result

mean_A = mean_even_idx(A)
assert mean_A == 63.0

mean_A
# Given an array A with shape (N,) make an array with all elements of A in reverse order
# and return as a matrix of size (N, 1).
A = np.arange(6)

def reverse(A):
    result = np.array([[]], dtype=A.dtype)
    # START ANSWER
    for i in reversed(A):
        result = np.append(result,[i]) 
    # END ANSWER
    return result.reshape(result.shape[0],1)

rev = reverse(A)
assert (rev == np.array([[5],[4],[3],[2],[1],[0]])).all()
rev<jupyter_output><empty_output><jupyter_text>Two dimensional data arraysIn this course you will be working a lot with matrices and vectors. The following exercises will let you practice with those.Given is data matrix X with shape (m, n)<jupyter_code>m = n = 5
X = np.arange(m * n).reshape(m, n)

print(X)
# Select the j-th column from the matrix X. What happens if you use X[j]? Is this correct?
j = 3
column = None
# START ANSWER
column = X[:,j]
# END ANSWER

assert (column == np.array([ 3,  8, 13, 18, 23])).all()
column
kk = [[1,2],[3,4]]
k_col0 = [row[0] for row in kk]
print(k_col0)
# Given an ndarray X with shape (m,n), calculate the mean of each column.
# Try doing this without using np.mean or loops.
# Hint: try to sum up the entries and dividing them by the number of elements

means = []
# START ANSWER
for i in range(X.shape[1]):
    means = np.append(means,np.mean(X[:,i]))
# means = np.array([np.sum(X[:,0])/X.shape[0],
#                  np.sum(X[:,1])/X.shape[0],
#                  np.sum(X[:,2])/X.shape[0],
#                  np.sum(X[:,3])/X.shape[0],
#                  np.sum(X[:,4])/X.shape[0]])
# END ANSWER

assert (means == np.array([10., 11., 12., 13., 14.])).all()
means
print(X)
# Now subtract the mean vector you just calculated from all the rows in your matrix leading to the 
# data matrix X_0. Yes this can be done without a loop! Hint: look at array broadcasting.

X_0 = None
# START ANSWER
X_0 = X - means
# END ANSWER

assert (X_0 == np.array([[-10., -10., -10., -10., -10.],
                         [ -5.,  -5.,  -5.,  -5.,  -5.],
                         [  0.,   0.,   0.,   0.,   0.],
                         [  5.,   5.,   5.,   5.,   5.],
                         [ 10.,  10.,  10.,  10.,  10.]])
       ).all()
X_0
# Given column j, find the largest element and return the entire row of this element.
# Hint: look at the function np.argmax for this.
m = n = 5
X = np.random.rand(m, n)
j = 3

# START ANSWER
i = np.argmax(X[:,j], axis=0)
max_row = X[i,:]
# END ANSWER

# Please inspect visually whether your code returns the right row
X, max_row<jupyter_output><empty_output><jupyter_text>Advanced Linear Algebra In Python 3, the @ operator is introduced for matrix multiplication. Let A be an array of shape (m, n) and let B be an array of shape (n, k) then we can write A @ B for the matrix multiplication of A and B. Note that there is conceptual difference between a 1 dimensional array V of size (N,) and a vector V as we know it from linear algebra. In linear algebra, a vector with $N$ elements has dimensions $N \times 1$. A ‘vector’ V as a numpy array has shape (N,).<jupyter_code># Calculate the inner product of two vector v and w both of shape (N,).
# Validate your result by computing the dot product using multiply and sum operations.
v = np.arange(5)
w = np.arange(5, 10)

# START ANSWER
result1 = v @ w
result2 = np.sum(v * w)
# END ANSWER
assert(result1 == result2)
print(result1, result2)
# Calculate the product of a matrix A of shape (M,N) with a vector v of shape (N,).
m = n = 5
X = np.arange(m * n).reshape(m, n)
v = np.arange(n)

product = None
# START ANSWER
product = X @ v
# END ANSWER

assert (product == np.array([ 30,  80, 130, 180, 230])).all()
product<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10056558039321303, 'fraction_numerical': 0.04858604901696741, 'mean_word_length': 3.916843220338983, 'pattern_counts': {'":': 0, '<': 97, '<?xml version=': 0, '>': 95, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 18, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '27', 'dir': 'jupyter-structured-clean-dedup', 'id': '389012', 'license': 'Apache-2.0', 'n_tokens_mistral': 6779, 'n_tokens_neox': 6029, 'n_words': 2789, 'path': 'week1/intro2-numpy.ipynb', 'repo_name': 'Timo9Madrid7/CSE2510_WeeklyLab', 'size': '18565'}