<jupyter_start><jupyter_text>$$\newcommand\bs[1]{\boldsymbol{1}}$$  IntroductionWe will see some very important concepts in this chapter. The dot product is used in every equation explaining data science algorithms so it's worth the effort to understand it. Then we will see some properties of this operation. Finally, we will to get some intuition on the link between matrices and systems of linear equations.  2.2 Multiplying Matrices and VectorsThe standard way to multiply matrices is not to multiply each element of one with each element of the other (this is called the *element-wise product*), but to calculate the sum of the products between rows and columns. The matrix product, also called **dot product**, is calculated as following:The dot product between a matrix and a vectorThe number of columns of the first matrix must be equal to the number of rows of the second matrix. Thus, if the shape of the first matrix is ($m \times n$), then the second matrix must be of shape ($n \times x$). The resulting matrix will then have the shape ($m \times x$).  The dot product of a matrix and a vector.$$\bs{A} \times \bs{b} = \bs{C}$$with $\bs{A}=\begin{bmatrix}    1 & 2\\\\    3 & 4\\\\    5 & 6\end{bmatrix}$ and $\bs{b}=\begin{bmatrix}    2\\\\    4\end{bmatrix}$. We saw that the formula is the following:$$\begin{align*}&\begin{bmatrix}    A_{1,1} & A_{1,2} \\\\    A_{2,1} & A_{2,2} \\\\    A_{3,1} & A_{3,2}\end{bmatrix}\times\begin{bmatrix}    B_{1,1} \\\\    B_{2,1}\end{bmatrix}=\\\\&\begin{bmatrix}    A_{1,1}B_{1,1} + A_{1,2}B_{2,1} \\\\    A_{2,1}B_{1,1} + A_{2,2}B_{2,1} \\\\    A_{3,1}B_{1,1} + A_{3,2}B_{2,1}\end{bmatrix}\end{align*}$$So we will have:$$\begin{align*}&\begin{bmatrix}    1 & 2 \\\\    3 & 4 \\\\    5 & 6\end{bmatrix}\times\begin{bmatrix}    2 \\\\    4\end{bmatrix}=\\\\&\begin{bmatrix}    1 \times 2 + 2 \times 4 \\\\    3 \times 2 + 4 \times 4 \\\\    5 \times 2 + 6 \times 4\end{bmatrix}=\begin{bmatrix}    10 \\\\    22 \\\\    34\end{bmatrix}\end{align*}$$To drive the point home, note the shapes of the matrices. We can see in this example that the shape of $\bs{A}$ is ($3 \times 2$) and the shape of $\bs{b}$ is ($2 \times 1$). So the shape of $\bs{C}$ must be ($3 \times 1$).  The dot product with numpyThe numpy function `dot()` can be used to compute the matrix product (or dot product). Let's reproduce the exemple above:<jupyter_code>A = np.array([[1, 2], [3, 4], [5, 6]])
A
B = np.array([[2], [4]])
B
C = np.dot(A, B)
C<jupyter_output><empty_output><jupyter_text>It is equivalent to use the method `dot()` of Numpy arrays:<jupyter_code>C = A.dot(B)
C<jupyter_output><empty_output><jupyter_text>The dot product of two matrices.$$\bs{A} \times \bs{B} = \bs{C}$$with:$$\bs{A}=\begin{bmatrix}    1 & 2 & 3 \\\\    4 & 5 & 6 \\\\    7 & 8 & 9 \\\\    10 & 11 & 12\end{bmatrix}$$and:$$\bs{B}=\begin{bmatrix}    2 & 7 \\\\    1 & 2 \\\\    3 & 6\end{bmatrix}$$So we have:$$\begin{align*}&\begin{bmatrix}    1 & 2 & 3 \\\\    4 & 5 & 6 \\\\    7 & 8 & 9 \\\\    10 & 11 & 12\end{bmatrix}\times\begin{bmatrix}    2 & 7 \\\\    1 & 2 \\\\    3 & 6\end{bmatrix}=\\\\&\begin{bmatrix}    2 \times 1 + 1 \times 2 + 3 \times 3 & 7 \times 1 + 2 \times 2 + 6 \times 3 \\\\    2 \times 4 + 1 \times 5 + 3 \times 6 & 7 \times 4 + 2 \times 5 + 6 \times 6 \\\\    2 \times 7 + 1 \times 8 + 3 \times 9 & 7 \times 7 + 2 \times 8 + 6 \times 9 \\\\    2 \times 10 + 1 \times 11 + 3 \times 12 & 7 \times 10 + 2 \times 11 + 6 \times 12 \\\\\end{bmatrix}\\\\&=\begin{bmatrix}    13 & 29 \\\\    31 & 74 \\\\    49 & 119 \\\\    67 & 164\end{bmatrix}\end{align*}$$Let's check the result with Numpy:<jupyter_code>A = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]])
A
B = np.array([[2, 7], [1, 2], [3, 6]])
B
C = A.dot(B)
C<jupyter_output><empty_output><jupyter_text>Formalization of the dot productAlgebraically, the dot product is the sum of the products of the corresponding entries of two sequences of numbers.$$C_{i,j} = A_{i,k}B_{k,j} = \sum_{k}A_{i,k}B_{k,j}$$At this point, your gut might be telling you that the dot product is a convoluted way to multiply two matrices. Your intuition for the best approach probably points toward simple element-wise multiplication (like we did with addition in the last lesson). To show you the value of the dot product, let's look at a real-life example of why we multiply matrices with the dot product method.  Exercise:  Calculating Weekly Pie Sales with the Dot ProductSay you own a pie shop and sell the following three types of pies: - Apple pies cost 3 dollars each - Cherry pies cost 4 dollars each - Blueberry pies cost 2 dollars eachAnd this table tells you many you've sold in the past 4 days:To calculate total sales for Monday, you'd do this:>Apple pie value + Cherry pie value + Blueberry pie value>or, mathematically...>$3 \times 13 + 4 \times 8 + 2 \times 6 = 83$By matching each price to each quantity, multiplying, and then summing all of the results, we can get our total sales. That process is actually the dot product!To prove this, create in the cell below a vector $\bs{p}$ and a matrix $\bs{S}$ containing the prices and sales data, respectively. Then, use numpy to calculate the dot product and determine total pie sales for each day. Finally, sum those daily sales to get the total sales.<jupyter_code># Your code here!!!<jupyter_output><empty_output><jupyter_text>Properties of the dot productWe will now see some interesting properties that arise from matrix multiplication. Knowing - or at least accepting - these properties will be useful in coming chapters. Matrix mutliplication is distributive$$\bs{A}(\bs{B}+\bs{C}) = \bs{AB}+\bs{AC}$$ Example$$\bs{A}=\begin{bmatrix}    2 & 3 \\\\    1 & 4 \\\\    7 & 6\end{bmatrix}, \bs{B}=\begin{bmatrix}    5 \\\\    2\end{bmatrix}, \bs{C}=\begin{bmatrix}    4 \\\\    3\end{bmatrix}$$$$\begin{align*}\bs{A}(\bs{B}+\bs{C})&=\begin{bmatrix}    2 & 3 \\\\    1 & 4 \\\\    7 & 6\end{bmatrix}\times\left(\begin{bmatrix}    5 \\\\    2\end{bmatrix}+\begin{bmatrix}    4 \\\\    3\end{bmatrix}\right)=\begin{bmatrix}    2 & 3 \\\\    1 & 4 \\\\    7 & 6\end{bmatrix}\times\begin{bmatrix}    9 \\\\    5\end{bmatrix}\\\\&=\begin{bmatrix}    2 \times 9 + 3 \times 5 \\\\    1 \times 9 + 4 \times 5 \\\\    7 \times 9 + 6 \times 5\end{bmatrix}=\begin{bmatrix}    33 \\\\    29 \\\\    93\end{bmatrix}\end{align*}$$is equivalent to$$\begin{align*}\bs{A}\bs{B}+\bs{A}\bs{C} &= \begin{bmatrix}    2 & 3 \\\\    1 & 4 \\\\    7 & 6\end{bmatrix}\times\begin{bmatrix}    5 \\\\    2\end{bmatrix}+\begin{bmatrix}    2 & 3 \\\\    1 & 4 \\\\    7 & 6\end{bmatrix}\times\begin{bmatrix}    4 \\\\    3\end{bmatrix}\\\\&=\begin{bmatrix}    2 \times 5 + 3 \times 2 \\\\    1 \times 5 + 4 \times 2 \\\\    7 \times 5 + 6 \times 2\end{bmatrix}+\begin{bmatrix}    2 \times 4 + 3 \times 3 \\\\    1 \times 4 + 4 \times 3 \\\\    7 \times 4 + 6 \times 3\end{bmatrix}\\\\&=\begin{bmatrix}    16 \\\\    13 \\\\    47\end{bmatrix}+\begin{bmatrix}    17 \\\\    16 \\\\    46\end{bmatrix}=\begin{bmatrix}    33 \\\\    29 \\\\    93\end{bmatrix}\end{align*}$$<jupyter_code>A = np.array([[2, 3], [1, 4], [7, 6]])
A
B = np.array([[5], [2]])
B
C = np.array([[4], [3]])
C<jupyter_output><empty_output><jupyter_text>$\bs{A}(\bs{B}+\bs{C})$:<jupyter_code>D = A.dot(B+C)
D<jupyter_output><empty_output><jupyter_text>is equivalent to $\bs{AB}+\bs{AC}$:<jupyter_code>D = A.dot(B) + A.dot(C)
D<jupyter_output><empty_output><jupyter_text>Matrix mutliplication is associative$$\bs{A}(\bs{BC}) = (\bs{AB})\bs{C}$$<jupyter_code>A = np.array([[2, 3], [1, 4], [7, 6]])
A
B = np.array([[5, 3], [2, 2]])
B<jupyter_output><empty_output><jupyter_text>$\bs{A}(\bs{BC})$:<jupyter_code>D = A.dot(B.dot(C))
D<jupyter_output><empty_output><jupyter_text>is equivalent to $(\bs{AB})\bs{C}$:<jupyter_code>D = (A.dot(B)).dot(C)
D<jupyter_output><empty_output><jupyter_text>But matrix multiplication is not necessarily* commutative$$\bs{AB} \neq \bs{BA}$$*An exception is when one matrix is the Identity Matrix. We'll learn about this in a later lesson.<jupyter_code>A = np.array([[2, 3], [6, 5]])
A
B = np.array([[5, 3], [2, 2]])
B<jupyter_output><empty_output><jupyter_text>$\bs{AB}$:<jupyter_code>AB = np.dot(A, B)
AB<jupyter_output><empty_output><jupyter_text>is different from $\bs{BA}$:<jupyter_code>BA = np.dot(B, A)
BA<jupyter_output><empty_output><jupyter_text>However vector multiplication is commutative$$\bs{x^{ \text{T}}y} = \bs{y^{\text{T}}x} $$<jupyter_code>x = np.array([[2], [6]])
x
y = np.array([[5], [2]])
y<jupyter_output><empty_output><jupyter_text>$\bs{x^\text{T}y}$:<jupyter_code>x_ty = x.T.dot(y)
x_ty<jupyter_output><empty_output><jupyter_text>is equivalent to $\bs{y^\text{T}x}$:<jupyter_code>y_tx = y.T.dot(x)
y_tx<jupyter_output><empty_output><jupyter_text>ExerciseUse numpy to show that$$(\bs{AB})^{\text{T}} = \bs{B}^\text{T}\bs{A}^\text{T}$$$\bs{A}$ and $\bs{B}$ are defined for you below.<jupyter_code>A = np.array([[2, 3], [1, 4], [7, 6]])
A
B = np.array([[5, 3], [2, 2]])
B<jupyter_output><empty_output><jupyter_text>$(\bs{AB})^{\text{T}}$:<jupyter_code># Complete the code below
AB_t = 
AB_t<jupyter_output><empty_output><jupyter_text>$\bs{B}^\text{T}\bs{A}^\text{T}$:<jupyter_code># Complete the code below
B_tA = 
B_tA<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.18921832884097034, 'fraction_numerical': 0.041617250673854446, 'mean_word_length': 3.412940057088487, 'pattern_counts': {'":': 0, '<': 77, '<?xml version=': 0, '>': 80, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '311680', 'license': 'Unlicense', 'n_tokens_mistral': 3740, 'n_tokens_neox': 3321, 'n_words': 1043, 'path': 'Learn Math/3. Linear Algebra/3.2 Multiplying Matrices and Vectors/3.2 Multiplying Matrices and Vectors.ipynb', 'repo_name': 'mcallistercs/learning-data-science', 'size': '9275'}