<jupyter_start><jupyter_text>Linear Algebra - Matrices---- Author: <NAME>- GitHub: [github.com/diegoinacio](https://github.com/diegoinacio)- Notebook: [linear-algebra_matrices.ipynb](https://github.com/diegoinacio/machine-learning-notebooks/blob/master/Mathematical-Foundations/linear-algebra_matrices.ipynb)---Linear Algebra topic about *Matrices*.<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>1. What is a matrix?---A matrix $ A$ is a *ractangular array* of elements arranged in *rows* $m$ and *columns* $n$.$$ \largeA = \begin{bmatrix}a_{11} & a_{12} & \cdots & a_{1n} \\a_{21} & a_{22} & \cdots & a_{2n} \\a_{31} & a_{32} & \cdots & a_{3n} \\\vdots & \vdots & \ddots & \vdots \\a_{m1} & a_{m2} & \cdots & a_{mn} \\\end{bmatrix}$$A *square matrix* is when the number of *rows* is the same of the number of *columns*.  2. Matrix operations---All the following operations are applicable to any vector in $\large \mathbb{R}^{m \times n}$. 2.1. Matrix addition---The addition of two matrices $\large M_{3\times2}$ and $\large N_{3\times2}$ is done by the sum of their correspondent components, resulting in another matrix.$$ \largeM+N = \begin{bmatrix}m_{11} & m_{12} \\m_{21} & m_{22} \\m_{31} & m_{32} \\\end{bmatrix} +\begin{bmatrix}n_{11} & n_{12} \\n_{21} & n_{22} \\n_{31} & n_{32} \\\end{bmatrix} =\begin{bmatrix}m_{11}+n_{11} & m_{12}+n_{12} \\m_{21}+n_{21} & m_{22}+n_{22} \\m_{31}+n_{31} & m_{32}+n_{32} \\\end{bmatrix}$$For example:$$ \largeM =\begin{bmatrix}2 & 5 \\3 & 7 \\8 & 6\end{bmatrix}\quad , \quadN =\begin{bmatrix}1 & 7 \\5 & 6 \\2 & 9\end{bmatrix}$$$$ \largeM + N =\begin{bmatrix}2+1 & 5+7 \\3+5 & 7+6 \\8+2 & 6+9\end{bmatrix} =\begin{bmatrix}3 & 12 \\8 & 13 \\10 & 15\end{bmatrix}$$<jupyter_code>M = np.array([
    [2, 5],
    [3, 7],
    [8, 6],
])
N = np.array([
    [1, 7],
    [5, 6],
    [2, 9],
])
print(M, end=" = M\n\n")
print(N, end=" = N\n\n")
print(M + N, end=" = M + N")<jupyter_output>[[2 5]
 [3 7]
 [8 6]] = M

[[1 7]
 [5 6]
 [2 9]] = N

[[ 3 12]
 [ 8 13]
 [10 15]] = M + N<jupyter_text>Properties of matrix addition---1. $M + (N + P) = (M + N) + P$2. $M + 0 = 0 + M = M$3. $M + N = N + M$  2.2. Matrix subtraction---Similarly to addition, the subtraction of two matrices $\large M$ and $\large N$ is done by the subtraction of their correspondent components, resulting in another matrix.$$ \largeM-N = \begin{bmatrix}m_{11} & m_{12} \\m_{21} & m_{22} \\m_{31} & m_{32} \\\end{bmatrix} -\begin{bmatrix}n_{11} & n_{12} \\n_{21} & n_{22} \\n_{31} & n_{32} \\\end{bmatrix} =\begin{bmatrix}m_{11}-n_{11} & m_{12}-n_{12} \\m_{21}-n_{21} & m_{22}-n_{22} \\m_{31}-n_{31} & m_{32}-n_{32} \\\end{bmatrix}$$For example:$$ \largeM =\begin{bmatrix}2 & 5 \\3 & 7 \\8 & 6\end{bmatrix}\quad , \quadN =\begin{bmatrix}1 & 7 \\5 & 6 \\2 & 9\end{bmatrix}$$$$ \largeM - N =\begin{bmatrix}2-1 & 5-7 \\3-5 & 7-6 \\8-2 & 6-9\end{bmatrix} =\begin{bmatrix}1 & -2 \\-2 & 1 \\6 & -3\end{bmatrix}$$<jupyter_code>M = np.array([
    [2, 5],
    [3, 7],
    [8, 6],
])
N = np.array([
    [1, 7],
    [5, 6],
    [2, 9],
])
print(M, end=" = M\n\n")
print(N, end=" = N\n\n")
print(M - N, end=" = M - N")<jupyter_output>[[2 5]
 [3 7]
 [8 6]] = M

[[1 7]
 [5 6]
 [2 9]] = N

[[ 1 -2]
 [-2  1]
 [ 6 -3]] = M - N<jupyter_text>Properties of matrix subtraction---1. $-(-M) = M$2. $-M + M = M - M = 0$3. $M - N \neq N - M$  2.3. Scalar multiplication---The scalar multiplication is the elementwise multiplication by a scalar number $\large \alpha$. The same rule can be applied to divisions.$$ \large\alpha M = \alpha \cdot\begin{bmatrix}m_{11} & m_{12} & \cdots & m_{1n} \\m_{21} & m_{22} & \cdots & m_{2n} \\m_{31} & m_{32} & \cdots & m_{3n} \\\vdots & \vdots & \ddots & \vdots \\m_{m1} & m_{m2} & \cdots & m_{mn}\end{bmatrix} =\begin{bmatrix}\alpha \cdot m_{11} & \alpha \cdot m_{12} & \cdots & \alpha \cdot m_{1n} \\\alpha \cdot m_{21} & \alpha \cdot m_{22} & \cdots & \alpha \cdot m_{2n} \\\alpha \cdot m_{31} & \alpha \cdot m_{32} & \cdots & \alpha \cdot m_{3n} \\\vdots & \vdots & \ddots & \vdots \\\alpha \cdot m_{m1} & \alpha \cdot m_{m2} & \cdots & \alpha \cdot m_{mn}\end{bmatrix}$$For example:$$ \large\alpha = 2\quad , \quadM =\begin{bmatrix}2 & 5 \\3 & 7 \\8 & 6\end{bmatrix}$$$$ \large\alpha \cdot M =\begin{bmatrix}2 \cdot 2 & 2 \cdot 5 \\2 \cdot 3 & 2 \cdot 7 \\2 \cdot 8 & 2 \cdot 6\end{bmatrix} =\begin{bmatrix}4 & 10 \\6 & 14 \\16 & 12\end{bmatrix}$$<jupyter_code>a = 2
M = np.array([
    [2, 5],
    [3, 7],
    [8, 6],
])
print(a, end=" = a\n\n")
print(M, end=" = M\n\n")
print(a*M, end=" = a.M")<jupyter_output>2 = a

[[2 5]
 [3 7]
 [8 6]] = M

[[ 4 10]
 [ 6 14]
 [16 12]] = a.M<jupyter_text>Properties of scalar multiplication---1. $(\alpha \beta) M = \alpha (\beta M)$2. $(\alpha + \beta) M = \alpha M + \beta M$3. $\alpha(M + N) = \alpha M + \alpha N$4. $1M = M$  2.4. Matrix multiplication---The multiplication of two matrices $\large M_{m \times n}$ and $\large N_{n \times p}$ is defined under the rule that the number of columns of the *first matrix* (${m \times n}$) must be equal to the number of rows of the *second matrix* (${n \times p}$), resulting in another matrix (${m \times p}$).$$ \large[MN]_{ij}=m_{i1} \cdot n_{1j}+m_{i2} \cdot n_{2j}+ \cdots + m_{in} \cdot n_{nj}=\sum_1^n m_{in} \cdot m_{nj}$$For example:$$ \largeM =\begin{bmatrix}2 & 3 & 8 \\5 & 7 & 6\end{bmatrix}\quad , \quadN =\begin{bmatrix}1 & 7 \\5 & 6 \\2 & 9\end{bmatrix}$$$$ \largeM \times N =\begin{bmatrix}2 \cdot 1 + 3 \cdot 5 + 8 \cdot 2 & 2 \cdot 7 + 3 \cdot 6 + 8 \cdot 9 \\5 \cdot 1 + 7 \cdot 5 + 6 \cdot 2 & 5 \cdot 7 + 7 \cdot 6 + 6 \cdot 9\end{bmatrix} =\begin{bmatrix}33 & 104 \\52 & 131\end{bmatrix}$$<jupyter_code>M = np.array([
    [2, 3, 8],
    [5, 7, 6]
])
N = np.array([
    [1, 7],
    [5, 6],
    [2, 9],
])
print(M, end=" = M\n\n")
print(N, end=" = N\n\n")
print(np.dot(M, N), end=" = M x N")<jupyter_output>[[2 3 8]
 [5 7 6]] = M

[[1 7]
 [5 6]
 [2 9]] = N

[[ 33 104]
 [ 52 131]] = M x N<jupyter_text>Properties of matrix multiplication---1. $M_{m \times n} N_{n \times m} \neq N_{n \times m} M_{m \times n}$2. $(M_{m \times n} N_{n \times p}) P_{p \times q} = M_{m \times n}(N_{n \times p} P_{p \times q})$3. $(M_{m \times n} + N_{m \times n}) P_{n \times p} = M_{m \times n}P_{n \times p} + N_{m \times n}P_{n \times p}$4. $P_{m \times n} (M_{n \times p} + N_{n \times p}) = P_{m \times n} M_{n \times p} + P_{m \times n} N_{n \times p}$5. $(\alpha M_{m \times n}) N_{n \times p} = M_{m \times n} (\alpha N_{n \times p}) = \alpha (M_{m \times n} N_{n \times p})$  3. Matrix transpose---The *transposition* of a matrix is the operation which converts rows into columns and vice versa, so $M_{m \times n}^T = M_{n \times m}$.For example:$$ \large\begin{bmatrix}1 & 7 \\5 & 6 \\2 & 9\end{bmatrix}^T =\begin{bmatrix}1 & 5 & 2 \\7 & 6 & 9\end{bmatrix}$$<jupyter_code>M = np.array([
    [1, 7],
    [5, 6],
    [2, 9],
])
print(M, end=" = M\n\n")
print(M.T, end=" = MT")<jupyter_output>[[1 7]
 [5 6]
 [2 9]] = M

[[1 5 2]
 [7 6 9]] = MT<jupyter_text>Properties of matrix transpose---1. $(M + N)^T = M^T + N^T$2. $(\alpha M)^T = \alpha M^T$3. $(M^T)^T = M$4. $(M_{m \times n} N_{n \times p})^T = N_{n \times p}^T M_{m \times n}^T$  4. Matrix determinant---The determinant of a *square matrix* ($\det(M)$ or $|M|$) is an operation which encodes certain properties of the linear transformation described by the matrix, resulting in a *real number*.$$ \large\det (M_{2 \times 2}) =\det \begin{bmatrix}m_{11} & m_{12} \\m_{21} & m_{22}\end{bmatrix} =m_{11} m_{22} - m_{12} m_{21}$$The standard method for a $3 \times 3$ matrix (or more) is based in a recursive process that gets the first row elements and multiply by the determinant of the $2 \times 2$ matrix (or more) that is not in the elements's row or column. Another pattern is the alternance of the operators $+ -$.$$ \large\begin{aligned}|M_{3 \times 3}| & =\det \begin{bmatrix}m_{11} & m_{12} & m_{13} \\m_{21} & m_{22} & m_{23} \\m_{31} & m_{32} & m_{33}\end{bmatrix} \\|M_{3 \times 3}| & = + m_{11} \cdot\det \begin{bmatrix}m_{22} & m_{23} \\m_{32} & m_{33}\end{bmatrix}- m_{12} \cdot\det \begin{bmatrix}m_{21} & m_{23} \\m_{31} & m_{33}\end{bmatrix}+ m_{13} \cdot\det \begin{bmatrix}m_{21} & m_{22} \\m_{31} & m_{32}\end{bmatrix} \\|M_{3 \times 3}| & = m_{11} \cdot (m_{22} \cdot m_{33} - m_{23} \cdot m_{32})- m_{12} \cdot (m_{21} \cdot m_{33} - m_{23} \cdot m_{31})+ m_{13} \cdot (m_{21} \cdot m_{32} - m_{22} \cdot m_{31}) \\|M_{3 \times 3}| & =(m_{11} m_{22} m_{33} + m_{12} m_{23} m_{31} + m_{13} m_{21} m_{32}) - (m_{13} m_{22} m_{31} + m_{12} m_{21} m_{33} + m_{11} m_{23} m_{32})\end{aligned}$$For example:$$ \large\begin{aligned}M & =\begin{bmatrix}1 & 7 & 3 \\5 & 6 & 8 \\2 & 9 & 4\end{bmatrix} \\|M| & =1 \cdot \det \begin{bmatrix}6 & 8 \\9 & 4\end{bmatrix}- 7 \cdot \det \begin{bmatrix}5 & 8 \\2 & 4\end{bmatrix}+ 3 \cdot \det \begin{bmatrix}5 & 6 \\2 & 9\end{bmatrix} \\|M| & = 1 \cdot (6 \cdot 4 - 8 \cdot 9) - 7 \cdot (5 \cdot 4 - 8 \cdot 2) + 3 \cdot (5 \cdot 9 - 6 \cdot 2) \\|M| & = - 48 - 28 + 99 \\|M| & = 23\end{aligned}$$There is a shortcut method that facilitates the computation and consists of expanding the elements horizontally and multiply the diagonals like in the matrix $2 \times 2$:$$ \large\begin{aligned}|M_{3 \times 3}| & =\det \begin{bmatrix}m_{11} & m_{12} & m_{13} \\m_{21} & m_{22} & m_{23} \\m_{31} & m_{32} & m_{33}\end{bmatrix}\begin{matrix}m_{11} & m_{12} \\m_{21} & m_{22} \\m_{31} & m_{32}\end{matrix} \\|M_{3 \times 3}| & =(m_{11} m_{22} m_{33} + m_{12} m_{23} m_{31} + m_{13} m_{21} m_{32}) - (m_{13} m_{22} m_{31} + m_{12} m_{21} m_{33} + m_{11} m_{23} m_{32})\end{aligned}$$<jupyter_code>M = np.array([
    [1, 7, 3],
    [5, 6, 8],
    [2, 9, 4],
])
print(M, end=" = M\n\n")
print(round(np.linalg.det(M)), end=" = |M|")<jupyter_output>[[1 7 3]
 [5 6 8]
 [2 9 4]] = M

23.0 = |M|<jupyter_text>Properties of matrix determinant---1. $|M^T| = |M|$2. $|MN| = |M| |N|$  5. Identity matrix---The *identity matrix* $I_n$ is a *square matrix* which the elements of the *main diagonal* is equal to 1 and all other elements is equal to 0.$$ \largeI_1 = \begin{bmatrix}1\end{bmatrix}\quad , \quadI_2 = \begin{bmatrix}1 & 0 \\0 & 1\end{bmatrix}\quad , \quadI_3 = \begin{bmatrix}1 & 0 & 0 \\0 & 1 & 0 \\0 & 0 & 1\end{bmatrix}\quad , \quadI_n = \begin{bmatrix}1 & 0 & \cdots & 0 \\0 & 1 & \cdots & 0 \\\vdots & \vdots & \ddots & \vdots \\0 & 0 & \cdots & 1\end{bmatrix}$$<jupyter_code>I2 = np.eye(2)
I3 = np.eye(3)
print(I2, end=" = I2\n\n")
print(I3, end=" = I3\n\n")<jupyter_output>[[1. 0.]
 [0. 1.]] = I2

[[1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]] = I3<jupyter_text>Properties of identity matrix---1. $M_{m \times n} I_n = I_m M_{m \times n} = M$2. $I_n^T = I_n$  6. Matrix inverse---A *square matrix* $M$ is invertible if it is *non-singular* and exists another square matrix $N$ which satisfies the following conditions:1. $MN = NM = I$3. Non-singular matrix is when $|M| \neq 0$$N$ is inverse the inverse of $M$ and is represented by $M^{-1}$, what it means that $M M^{-1} = I$.For example:if:$$ \largeM =\begin{bmatrix}2 & 1 & 0 \\0 & 1 & 0 \\1 & 2 & 1\end{bmatrix} $$so:$$ \largeM M^{-1} = I_3\quad \Rightarrow \quad\begin{bmatrix}2 & 1 & 0 \\0 & 1 & 0 \\1 & 2 & 1\end{bmatrix}\begin{bmatrix}m_{11} & m_{12} & m_{13} \\m_{21} & m_{22} & m_{23} \\m_{31} & m_{32} & m_{33}\end{bmatrix} =\begin{bmatrix}1 & 0 & 0 \\0 & 1 & 0 \\0 & 0 & 1\end{bmatrix}$$$$ \large\begin{bmatrix}2 m_{11} + m_{21} & 2 m_{12} + m_{22} & 2 m_{13} + m_{23} \\m_{21} & m_{22} & m_{23} \\m_{11} + 2 m_{21} + m_{31} & m_{12} + 2 m_{22} + m_{32} & m_{13} + 2 m_{23} + m_{33}\end{bmatrix} = \begin{bmatrix}1 & 0 & 0 \\0 & 1 & 0 \\0 & 0 & 1\end{bmatrix}$$After solving this system of equations (we can solve this by using Cramer's rule, for example), we have:$$ \largeM^{-1} = \begin{bmatrix}\frac{1}{2} & - \frac{1}{2} & 0 \\0 & 1 & 0 \\- \frac{1}{2} & - \frac{3}{2} & 1\end{bmatrix}$$<jupyter_code>M = np.array([
    [2, 1, 0],
    [0, 1, 0],
    [1, 2, 1],
])
print(M, end=" = M\n\n")
print(np.linalg.inv(M), end=" = |M|")<jupyter_output>[[2 1 0]
 [0 1 0]
 [1 2 1]] = M

[[ 0.5 -0.5  0. ]
 [ 0.   1.   0. ]
 [-0.5 -1.5  1. ]] = |M|
{'fraction_non_alphanumeric': 0.2409251790270804, 'fraction_numerical': 0.07992427360276566, 'mean_word_length': 3.0936657681940702, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 30, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '127375', 'license': 'MIT', 'n_tokens_mistral': 5793, 'n_tokens_neox': 5102, 'n_words': 1540, 'path': 'Mathematical-Foundations/linear-algebra_matrices.ipynb', 'repo_name': 'XGBTrain5/07.ml_diegoinacio', 'size': '12155'}