<jupyter_start><jupyter_text>Numpy allows us to create vectors rather than lists.<jupyter_code>#Take list 1, 2, 3 and make it a vector
x_vector = np.array([1,2,3])<jupyter_output><empty_output><jupyter_text>Vectors have *shape*, or *direction.*<jupyter_code>x_vector.shape<jupyter_output><empty_output><jupyter_text>Additionally, we can create matrices.<jupyter_code>matrix = np.array([[1, 2, 3],
                   [4, 5, 6]])
matrix<jupyter_output><empty_output><jupyter_text>Addition/Subtraction \begin{equation}    A+3=\begin{bmatrix}      a_{11} & a_{12} \\      a_{21} & a_{22}       \end{bmatrix}+3    =\begin{bmatrix}      a_{11}+3 & a_{12}+3 \\      a_{21}+3 & a_{22}+3       \end{bmatrix}\end{equation}\begin{equation}    A-3=\begin{bmatrix}      a_{11} & a_{12} \\      a_{21} & a_{22}       \end{bmatrix}-3    =\begin{bmatrix}      a_{11}-3 & a_{12}-3 \\      a_{21}-3 & a_{22}-3       \end{bmatrix}\end{equation} Match the rows/columns and add the scalar. And similarly for matrices:  \begin{equation}A_{2 \times 2} + B_{2 \times 2}= \begin{bmatrix}  a_{11}+b_{11} & a_{12}+b_{12} \\  a_{21}+b_{21} & a_{22}+b_{22}     \end{bmatrix}_{2 \times 2}\end{equation} Here's an example with `numpy`<jupyter_code>#Initial matrix has dimensions 2 x 3, this one should be 3 x 2
matrix2 = np.random.rand(3,2)

matrix*matrix2<jupyter_output><empty_output><jupyter_text>Scalar Multiplication   Matrix Multiplicataion \begin{align}    A_{3 \times 2} \times C_{2 \times 3}=&    \begin{bmatrix}      a_{11} & a_{12} \\      a_{21} & a_{22} \\      a_{31} & a_{32}       \end{bmatrix}_{3 \times 2}    \times    \begin{bmatrix}      c_{11} & c_{12} & c_{13} \\      c_{21} & c_{22} & c_{23}     \end{bmatrix}_{2 \times 3} \\    =&    \begin{bmatrix}      a_{11} c_{11}+a_{12} c_{21} & a_{11} c_{12}+a_{12} c_{22} & a_{11} c_{13}+a_{12} c_{23} \\      a_{21} c_{11}+a_{22} c_{21} & a_{21} c_{12}+a_{22} c_{22} & a_{21} c_{13}+a_{22} c_{23} \\      a_{31} c_{11}+a_{32} c_{21} & a_{31} c_{12}+a_{32} c_{22} & a_{31} c_{13}+a_{32} c_{23}    \end{bmatrix}_{3 \times 3}  \end{align} Requirements:* Columns of first matrix = Rows of second matrix\$$c_x=r_y$$Result:* A matrix of with the same amount of rows as the first matrix, and the columns of the second.$$r_x \times c_y$$Memory Devices* C for condition, c for columns first* R for result, r for rows first  Terminology **Basis vectors** the $\color{red}{columns}$ of the matrix, which are the $\color{red}{\mbox{input dimensions}}$ mapping to a certain output. **Linear Combination**:![](https://wikimedia.org/api/rest_v1/media/math/render/svg/0a701e9b8de6ac82da0fe68b425b44688d1d0b97) **Span** - all possible linear combinations.  So if I have $a_{1}\vec{v_{1}} + a_{1}\vec{v_{2}}$, then the span is the entire 2D coordinate plane, *unless they are linearly dependent* **Linear Dependence** - one of the vectors can be defined as a linear combination of the others.* Consider a third vector, $\vec{u}$.  It would be linear dependent if: $$\vec{u} = a\vec{v} + b\vec{v}$$* Basically, this would be having a vector that doesn't really do anything.  If I said, go 3 miles east, then four miles north, those vectors would be **linearly independent**.  But if I added on 'go five miles northeast,' that would be useless information, denoting **linear dependency**.  A **rank** is the dimensions of the output of a linear transformation.  It is equal to the number of columns in the column space.  When the rank is the same as the number of columns in the matrix, then it is spanning all possible dimensions, and it can be considered **full rank**A **column space** is the set of all possible outputs of a linear transformation, also known as the *span of the columns in the matrix* The **Null space**, or the **kernel**, of a vector space is all vector mappings $\vec{v}$ in  $$A\vec{x}=\vec{v}$$where the mapping is the 0 vector.  Everything in the column space that is *not in the kernel* is known as the image of $A$  Linear Transformations Essentially a function, taking in a vector as a parameter and outputting another vector.Two important properties:* Lines remain lines* Origin remains fixedTry to keep in mind that with any linear transformation in 2D, the *gridlines stay parallel and evenly spaced.* ![](https://i.imgur.com/gwEl6Rj.png) Looking at this, where the vector started at $(-1, 2)$, we can see that this is essentially a linear algebra distributive property. Commonly, this linear transformed $\hat{i} and \hat{j}$ is represented as follows: ![](https://i.imgur.com/4XAkDdE.png) And finally, we can represent the linear transformation as:  ![](https://i.imgur.com/cOPqkbV.png) Common transformation is a 90&deg; counterclockwise * $\hat{i}$ = \begin{bmatrix}0\\1\end{bmatrix} * $\hat{j}$ = \begin{bmatrix}-1\\0\end{bmatrix}   The big idea **All matrices can be thought of as linear transformations.**  Matrix Composition Multiple transformations can be thought of as one transformation, a **composition**, the product of two transformation matrices ![](https://i.imgur.com/pm12ct9.png) Note here that **order matters**.  The right-hand matrix (rotation) is the *first* transformation, and the left-hand (shear) is the *second* transformation.  It is similar to how in the equation $f(g(x))$, $g(x)$ is evaluated first. With that said, matrix multiplication *is* associative.  That is, if we have matrices $A, B \mbox{ and } C$$$A(BC) = (AB)C$$because we are just applying three transformations in the same order  Determinants The **scaled area** enclosed by vector components $\hat{i}\mbox{ and }\hat{j}$ is called the **determinant*** For 3D planes, the **scaled volume** is the determinant, a **parallelepiped** Determinants of *linearly dependent* vectors will be zero.  Thus, solving the determinant is a way to solve for the linear dependence of two vectors. When the **orientation** of space has been **inverted** ($\hat{i}\mbox{ now on the right of }\hat{j}$, the **determinant is negative**   Systems of Equations ![](https://i.imgur.com/oDtl6IR.png) Commonly, the matrix of the coefficients is called $A$, the variable vector is referred to as $\vec{x}$, and the vector of the constants is known as $\vec{v}$.  This gives us the equation:  $$A\vec{x}=\vec{v}$$ In words:  Applying the transformation $A$ to vector $\vec{x}$ makes $\vec{x}$ land on $\vec{v}$ If we want to go from $\vec{x}$ to $\vec{v}$, we have to divide the matrix $A$, or **multiply by the inverse matrix,** $A^{-1}$  Identity Matrix Multiplying the transformation by the inverse transformation, $A\times A^{-1}$, results in no change at all.  This is called the identity matrix, and has form:\begin{equation*}\mbox{$A\times A^{-1}$} = \begin{bmatrix}1 & 0 \\0 & 1\end{bmatrix}\end{equation*} If the determinant is zero, than there **is no inverse matrix**, $A^{-1}$.  In 2D, this is when the output is squished on a line.  In 3D, it is squished onto a plane.  These have **ranks** of 1, and 2 respectively. A **rank** is therefore the dimensions of the output of a linear transformation. A **column space** is the set of all possible outputs of a linear transformation.  Nonsquare Matrices Consider a $2 \times 3$ matrix, $B$ \begin{equation*}B = \begin{bmatrix}a & b \\d & e \\g & h\end{bmatrix}\end{equation*} This would mean we have a *2D input* mapping to a *3d output*.  The number of columns map to the input, and the number of rows to the output.  Dot Product ![](https://i.imgur.com/FD5UZON.png) ![](https://i.imgur.com/2IsfhnH.png)  Properties Thinking in terms of projections help explain these.* Perpendicular:  Dot product is *zero** Similar direction: Dot product is *positive** Opposite direction: Dot product is *negative*  Cross Product ![](https://i.imgur.com/AISfIbb.png) Sign changes, order matters.  Remember it with $\hat{i}\mbox{ and }\hat{j}$.  Their dot product, with $\hat{i}$ on the left of the cross product multiplication, would be positive.Big idea with sign? - If the first vector is on the left of the second, then the result will be positive. ![](https://i.imgur.com/4ncheEI.png)  Properties:  * As vectors get closer to being perpendicular, the result increases* The result is a vector* Perpendicular to the parallelogram formed, considering the right hand rule  Calculation It is the determinant, but with the first column being the basis vectors. ![](https://i.imgur.com/jNzBH5d.png) Remember that this involves striking through the rows and then carrying out the determinant like usual.  Duality Any transformation from a 2D plane to the 1D number line can be equivalently thought of as a linear transformation (a $1\times 2$ matrix) or a vector (a $2 \times1$ cell) The vector equivalent to this linear transformation is called the **dual vector** ![](https://i.imgur.com/9dpstM9.png)  Cramer's Rule ![](https://i.imgur.com/5jujVop.png) The "logic" behind this is that areas get scaled by the same transformation, $A$.  See [this video](https://www.youtube.com/watch?v=jBsC34PxzoM&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab&index=12) for more information.  Change of Basis We use the basis vectors $\hat{i}\mbox{ and }\hat{j}$, but we can change these and then scale them with vectors. Translating between coordinate systems uses a matrix translator of sorts, called the **change of basis matrix**. ![](https://i.imgur.com/lxzj4iJ.png) Multiplying the change of basis matrix written in our coordinates by the vector in the other coordinates gives the same vector in our coordinates.  If we used the inverse of the change of basis matrix instead and multiplied by our coordinates, we could translate our coordinates to the other coordinates. $$A^{-1}MA$$ The above equation generally suggests a coordinate system shift.  Computing this, where $A$ is the change of basis matrix and $M$ is the transformation will result in the matrix that can translate a vector in the other coordinate system in the **same way** as it translates one in our coordinate system.    Eigenvectors Vectors that remain on their same span when transformed.  They are only stretched or shrunk, but **not** knocked off of their span path. ![](https://i.imgur.com/wGLkAxW.png) To turn both sides into matrix-vector multiplication, we multiply $\lambda$ by the identity matrix, $I$, which brings us to this: ![](https://i.imgur.com/iSCU6wK.png) Often, it turns out this can be calculated by calculating the roots of a polynomial.  Here, we can see an example of this, showing that a matrix \begin{equation*}B = \begin{bmatrix}0 & -1\\1 & 0 \\\end{bmatrix}\end{equation*}(the rotation matrix) has no eigenvectors. ![](https://i.imgur.com/llAvLzs.png) On the other hand, we can also have times where the diagonals of the matrix are the eigenvalues themselves.  This happens with diagonal matrices. ![](https://i.imgur.com/2XBiVCK.png) Change of basis is used in conjunction with eigenvectors and diagonal matrices to simplify computations, rerouting the coordinate system to one with eigenvectors.  Linearity ![](https://i.imgur.com/qBNpdNM.png)<jupyter_code>from IPython.core.display import HTML
def css_styling():
    styles = open("../../jupyter-styles/custom.css", "r").read()
    return HTML(styles)
css_styling()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12475424486148347, 'fraction_numerical': 0.025379803395889185, 'mean_word_length': 4.385466794995188, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 20, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '177250', 'license': 'MIT', 'n_tokens_mistral': 3560, 'n_tokens_neox': 3282, 'n_words': 1422, 'path': '_posts/ml/Essence-of-Linear-Algebra.ipynb', 'repo_name': 'luke-anglin/lukes_site', 'size': '11190'}