<jupyter_start><jupyter_text>Lists vs. Arrays<jupyter_code>import numpy as np
L = [1,2,3]                 # creating python list
A = np.array([1,2,3])       # creating numpy array
for e in L:
    print e
for e in A:
    print e
L.append(4)        # adding element to list
L = L + [5]        # another way to add element to list
L
A + A              # adding two arrays element wise addition in numpy (vector addition)
A * 2              # multiplying vector times a scalar
A**2               # squaring a vector<jupyter_output><empty_output><jupyter_text>Something to keep in mind about numpy is that most functions act element wise. This just means that the function is applied to each element of the vector or matrix.<jupyter_code>np.sqrt(A)         # taking square root of all elements in vector 
np.log(A)          # element wise log
np.exp(A)          # element wise exponential<jupyter_output><empty_output><jupyter_text>With numpy you can treat lists like a vector, a mathematical object.  --- Dot ProductsRecall that there are two definitions of the dot product, and they are each equivalent. 1: The first is the summation of the element wise multiplication of the two vectors:$$a \cdot b = a^Tb = \sum_{d=1}^Da_db_d$$Here $d$ is being used to index each component. Notice that the convention $a^Tb$ implies that the vectors are column vectors, which means that the result is a (1 x 1), aka a scalar. 2: The second is the magnitude of $a$, times the magnitude of $b$, times the cosine of the angle between $a$ and $b$:$$a \cdot b = |a||b|cos\theta_{ab}$$This method is not very convenient unless we know each of the things on the right hand side to begin with. It would generally be used to find the angle itself.  Definition 1Let's look at this in code.<jupyter_code>a = np.array([1,2])
a
b = np.array([2,1])
b<jupyter_output><empty_output><jupyter_text>If we wanted to use the direct definition of the dot product, we would want to loop through both arrays simultaneously, multiply each corresponding element together, and add it to the final sum.<jupyter_code>dot = 0
for e, f in zip(a,b):
    dot += e*f
dot                      # result is 4 as expected<jupyter_output><empty_output><jupyter_text>Another interesting operation that you can do with numpy arrays is multiply two arrays together. We have already seen how to multiply a vector by a scalar.<jupyter_code>a * b             # element wise multiplication of a and b<jupyter_output><empty_output><jupyter_text>However, the above method could not be done with two arrays of different sizes. Now, if we summed the result of `a * b` we would end up with the dot product.<jupyter_code>np.sum(a * b)           # this is the element wise multiplication of a and b, summed<jupyter_output><empty_output><jupyter_text>An interesting thing about numpy is that the sum function is an instance method of the numpy array itself. So we could also write the above as:<jupyter_code>(a * b).sum()<jupyter_output><empty_output><jupyter_text>Now, while both of the above methods yield the correct answer, there is a more convenient way to calculate the dot product. Numpy comes packaged with a dot product function.<jupyter_code>np.dot(a,b)<jupyter_output><empty_output><jupyter_text>Like the `sum` function, the `dot` function is also an instance method of the numpy array, so we can call it on the object itself.<jupyter_code>a.dot(b)<jupyter_output><empty_output><jupyter_text>This is also equivalent to:<jupyter_code>b.dot(a)<jupyter_output><empty_output><jupyter_text>Definition 2Let's now look at the alternative definition of the dot product, to calculate the angle between $a$ and $b$. For this we need figure out how to calculate the length of a vector. We can do this by taking the square root of the sum of each element squared. In other words, use pythagorean theorem.<jupyter_code>amag = np.sqrt( (a * a).sum())
amag<jupyter_output><empty_output><jupyter_text>Numpy actually has a function to do all of this work for us, since it is such a common operation. It is part of the linalg module in numpy, which also contains many other linear algebra functions.<jupyter_code>amag = np.linalg.norm(a)
amag<jupyter_output><empty_output><jupyter_text>Now with this in hand, we are ready to calculate the angle. For clarity, the angle is defined as:$$cos\theta_{ab} = \frac{a \cdot b}{|a||b|}$$<jupyter_code>cosangle = a.dot(b) / (np.linalg.norm(a) * np.linalg.norm(b))
cosangle<jupyter_output><empty_output><jupyter_text>So the cosine of the angle is 0.8, and the actual angle is the arc cosine of 0.8:<jupyter_code>angle = np.arccos(cosangle)
angle<jupyter_output><empty_output><jupyter_text>By default this is in radians.  --- Vectors and MatricesA numpy array has already been shown to be like a vector: we can add them, multiply them by a scalar, and perform element wise operations like `log` or `sqrt`. So what is a matrix then? Think of it as a two dimensional array.<jupyter_code>M = np.array([ [1,2], [3,4] ])        # creating a matrix. 1st index is row, 2nd index is col
M
M[0][0]                      # one way of accessing values from matrix
M[0,0]                       # another shorthand way of accessing value in matrix<jupyter_output><empty_output><jupyter_text>There is an actual data type in numpy called matrix as well.<jupyter_code>M2 = np.matrix([ [1,2], [3,4] ])
M2<jupyter_output><empty_output><jupyter_text>This works somewhat similarly to a numpy array, but it is not exactly the same. Most of the time we just use numpy arrays, and in fact the official documentation actually recommends not using numpy matrix. If you see a matrix, it is a good idea to convert it into an array:<jupyter_code>M3 = np.array(M2)
M3<jupyter_output><empty_output><jupyter_text>Note that even though this is now an array, we still have convenient matrix operations. For example if we wanted to find the transpose of M:<jupyter_code>M
M.T<jupyter_output><empty_output><jupyter_text>To summarize, we have shown that a matrix is really just a 2-dimensional numpy array, and a vector is a 1-dimensional numpy array. So a matrix is really like a 2 dimensional vector. The more general way to think about this is that a matrix is a 2-dimensional mathematical object that contains numbers, and a vector is a 1-dimensional mathematical object that contains numbers. Sometimes you may see vectors represented as a 2-d object. For example, in a math textbook a column vector may be described as (3 x 1), and a row vector (1 x 3). Sometimes we may represent them like this in numpy.  --- Generating Matrices to Work WithSometimes we just need arrays to try stuff on, like in this course. One way to do this is to use `np.array` and pass in a list:<jupyter_code>np.array([1,2,3])<jupyter_output><empty_output><jupyter_text>However, this is inconvenient since each element needs to be typed in manually. What if we wanted arrays of different sizes?Lets start by creating a vector of zeros.<jupyter_code>Z = np.zeros(10)
Z<jupyter_output><empty_output><jupyter_text>We can also create a 10 x 10 matrix of all zeros.<jupyter_code>Z = np.zeros((10, 10))
Z<jupyter_output><empty_output><jupyter_text>Notice that the function still takes in 1 input, a tuple containing each dimension. There is an equivalent function that creates an array of all ones.<jupyter_code>O = np.ones((10, 10))
O<jupyter_output><empty_output><jupyter_text>What if we wanted random numbers? We could use `np.random.random`.<jupyter_code>R = np.random.random((10,10))
R<jupyter_output><empty_output><jupyter_text>One thing that we can quickly see is that all of these values are greater than 0 and less than 1. Whenever we talk about random numbers, you should be interested in the probability distribution that the random numbers came from. This particular random functions gives us uniformly distributed numbers between 0 and 1. What if we wanted gaussian distributed numbers? Numpy has a function for that as well.<jupyter_code># G = np.random.randn((10, 10))          this will not work, sine randn does not take tuple

G = np.random.randn(10,10)
G<jupyter_output><empty_output><jupyter_text>Numpy arrays also have convenient ways for us to calculate statistics of matrices.<jupyter_code>G.mean()      # gives us the mean
G.var()       # gives us the variance<jupyter_output><empty_output><jupyter_text>--- Matrix ProductsWhen you learn about matrix products in linear algebra, you generally learn about matrix multiplication. Matrix multiplication has a special requirement, and that is that the inner dimensions of the matrices you are multiplying must match. For example say we have matrix `A` that is **(2, 3)** and a matrix `B` that is **(3, 3)**, we can multiply A * B, since the inner dimension is 3, however we cannot multiply B * A, since the inner dimensions are 3 and 2, hence they do not match. Why do we have this requirement when we multiply matrices? Well lets look at the definition of matrix multiplication:$$C(i,j) = \sum_{k=1}^KA(i,k)B(k,j)$$So the (i,j)th entry of $C$ is the sum of the multiplication of all the corresponding elements of the ith row of A and the jth column of B. In other words, C(i,j) is the dot product of the ith row of A and the jth column of B. Because of this, we actually use the `dot` function in numpy! That does what we recognize as matrix multiplication! A very natural thing to want to do, both in math and in computing, is element by element multiplication! $$C(i,j) = A(i,j) * B(i,j)$$For vectors, we already saw that an asterisk `*` operation does this. As you may have guessed, for 2-d arrays, the asterisk also does element wise multiplication. That means that when you use the `*` on multidimensional arrays, both of them have to be the exact same size. This may seem odd, since in other languages, the asterisk does mean real matrix multiplication. So we just need to remember that in numpy, the asterisk `*` does mean element by element multiplication, and the `dot` means matrix multiplication. Another thing that is odd is that when we are writing down mathematical equations, there isn't even a well defined symbol for element wise multiplication. Sometimes researchers use a circle with a dot inside of it, sometimes they use a circle with an x inside of it. But there does not seem to be a standard way to do that in math.  --- More Matrix OperationsThe dot product is often referred to as the **inner product**. But we can also look at the **outer product**. An outer product is going to a be a **column vector** times a **row vector**. An inner product is going to a be a **row vector** times a **column vector**. For more information on this checkout my linear algebra walk through in the math appendix.<jupyter_code>a = np.array([1,2])
a
b = np.array([3,4])
b<jupyter_output><empty_output><jupyter_text>Lets first look at the dot product:<jupyter_code>np.dot(a,b)<jupyter_output><empty_output><jupyter_text>Now the inner product:<jupyter_code>np.inner(a,b)<jupyter_output><empty_output><jupyter_text>We see that it is the same as the dot product. Now let's look at the outer product:<jupyter_code>np.outer(a,b)<jupyter_output><empty_output><jupyter_text>We can get the same result if we ensure that out `a` and `b` are proper matrices, and then use the dot product. Note, here we see the equivalence to the `inner` and `outer` methods above.<jupyter_code>a = np.array([[1,2]])
a
b = np.array([[3,4]])
b
b = b.T
b
np.dot(a,b)
np.dot(b,a)<jupyter_output><empty_output><jupyter_text>Sum along certain axis If we want to sum along all of the rows in a matrix we can use the following:<jupyter_code>A = np.array([[1,1,1], [2,2,2], [3,3,3]])       
A.sum(axis=1)                                    # sum along each row, axis = 1 
A.sum(axis=0)                                    # sum along each column, axis = 0<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07982411635379671, 'fraction_numerical': 0.010147133434804667, 'mean_word_length': 4.440202391904323, 'pattern_counts': {'":': 0, '<': 125, '<?xml version=': 0, '>': 125, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '31', 'dir': 'jupyter-structured-clean-dedup', 'id': '507569', 'license': 'MIT', 'n_tokens_mistral': 3557, 'n_tokens_neox': 3431, 'n_words': 1628, 'path': 'notebooks/Programming Appendix/Numpy Overview.ipynb', 'repo_name': 'NathanielDake/NathanielDake.github.io', 'size': '11826'}