<jupyter_start><jupyter_text>VectorsInformally, we think of a vector as an object that has magnitude and direction. More formally, we think of an $n$-dimensional vector as an ordered tuple of numbers $(x_1, x_2, \ldots, x_n)$ that follows the rules of scalar multiplication and vector addition.<jupyter_code>%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Vector spaceA vector space is a collection of vectors which is closed under addition and scalar multiplication. Examples:- Euclidean plane $\mathbb{R}^2$ is a familiar vector space- The vector $\pmatrix{0 & 0}$ is a trivial vector space that is a **vector subspace** of Euclidean space.- Polynomial functions of order $k$ is a vector spacePolynomials of order 3 have the form $ax^3 + bx^2 + cx + d$, and can be represented as the vector$$\pmatrix{a \\ b \\ c \\ d}$$- The space of all continuous functions is a vector spaceConsider two continuous functions, say, $f(x) = x^2$ and $g(x) = x^3)$. Scalar multiplication $(2 f)(x) = 2x^2$ and addition  $(f + g)(x) = x^2 + x^3$ are well defined and the result is a continuous function, so the space of all continuous functions is also a vector space. In this case, it is an infinite-dimensional vector space. Vector spaces are important because the theorems of linear algebra apply to all vector spaces, not just Euclidean space.  Column vectorsWhen we describe a vector $x$, we mean the column vector.  The row vector is denoted $x^T$.<jupyter_code>x = np.random.random((5,1))
x
x.T<jupyter_output><empty_output><jupyter_text>Dimensions<jupyter_code>len(x)<jupyter_output><empty_output><jupyter_text>LengthThe length of a vector is the Euclidean norm (i.e. Pythagoras theorem)<jupyter_code>np.linalg.norm(x)
np.sqrt(np.sum(x**2))<jupyter_output><empty_output><jupyter_text>Direction<jupyter_code>n = x/np.linalg.norm(x)
n
np.linalg.norm(n)<jupyter_output><empty_output><jupyter_text>Norms and distancesRecall that the 'norm' of a vector $v$, denoted $||v||$ is simply its length.  For a vector with components $$v = \left(v_1,...,v_n\right)$$the norm of $v$ is given by:    $$||v|| = \sqrt{v_1^2+...+v_n^2}$$The distance between two vectors is the length of their difference:    $$d(v,w) = ||v-w||$$<jupyter_code>u = np.array([3,0]).reshape((-1,1))
v = np.array([0,4]).reshape((-1,1))
np.linalg.norm(u - v)
np.linalg.norm(v - u)
np.sqrt(np.sum((u - v)**2))<jupyter_output><empty_output><jupyter_text>Vector operations<jupyter_code>x = np.arange(3).reshape((-1,1))
y = np.arange(3).reshape((-1,1))<jupyter_output><empty_output><jupyter_text>Scalar multiplication<jupyter_code>3 * x<jupyter_output><empty_output><jupyter_text>Vector addition<jupyter_code>x + y<jupyter_output><empty_output><jupyter_text>Linear operations<jupyter_code>3*x + 4*y<jupyter_output><empty_output><jupyter_text>Transposition<jupyter_code>x.T<jupyter_output><empty_output><jupyter_text>Dot productThe dot product of two vectors $u$ and $v$ is written as $u \cdot v$ and its value is given by $u^Tv$. The dot product of two $n$ dimensional vectors $v$ and $w$ is given by:$$u \cdot v = u_1v_1+...+u_nv_n$$I.e. the dot product is just the sum of the product of the components.The inner product $\langle u,v \rangle$ of two vectors is a generalization of the dot product. It is any function that takes two vectors, returns a scalar (here we just consider inner products that return real numbers), and obeys the following properties:- symmetry $\langle u,v \rangle = \langle v,u \rangle$- positive definite     - $\langle v,v \rangle \ge 0$    - $\langle v,v \rangle = 0 \implies v = 0$- bilinear    - $\langle au,v \rangle = a \langle u,v \rangle$    - $\langle u + v,w \rangle = \langle u,w \rangle + \langle v,w \rangle$    - Linearity also applies to second argument because of symmetryAny inner product determines a norm via:$$||v|| = \langle v,v \rangle^{\frac12}$$  Calculating dot products<jupyter_code>u = np.array([3,3]).reshape((-1,1))
v = np.array([2,0]).reshape((-1,1))
np.dot(u.T, v)<jupyter_output><empty_output><jupyter_text>You can also use the `@` operator to do matrix multiplication<jupyter_code>u.T @ v
np.sum(u * v)<jupyter_output><empty_output><jupyter_text>Geometry of dot productGeometrically, the dot product is the product of the length of $v$ and the length of the projection of $u$ onto the unit vector $\widehat{v}$. $$u \cdot v = \lvert u \rvert \lvert v \rvert \cos \theta$$<jupyter_code>plt.plot(*zip(np.zeros_like(u), u), 'b-')
plt.text(1.8, 0.1, 'v', fontsize=14)
plt.plot(*zip(np.zeros_like(v), v), 'b-')
plt.text(2.8, 2.6, 'u', fontsize=14)
plt.tight_layout()
pass<jupyter_output><empty_output><jupyter_text>Angle between two vectors<jupyter_code>cos_angle = np.dot(u.T, v)/(np.linalg.norm(u)*np.linalg.norm(v))
cos_angle
theta = 180/np.pi*np.arccos(cos_angle)
theta<jupyter_output><empty_output><jupyter_text>Outer productNote that the inner product is just matrix multiplication of a $1\times n$ vector with an $n\times 1$ vector.  In fact, we may write:$$\langle v,w \rangle = v^Tw$$The *outer product* of two vectors is just the opposite. It is given by:$$v\otimes w = vw^T$$Note that I am considering $v$ and $w$ as *column* vectors.  The result of the inner product is a *scalar*. The result of the outer product is a *matrix*.For example, if $v$ and $w$ are both in $\mathbb{R}^3$$$v \otimes w = \pmatrix{v_1\\v_2\\v_3} \pmatrix{w_1 & w_2 & w_3} = \pmatrix{v_1w_1 & v_1w_2 & v_1w_3\\v_2w_1 & v_2w_2 & v_2w_3 \\ v_3w_1 & v_3w_2 & v_3w_3}$$<jupyter_code>v = np.array([1,2,3]).reshape((-1,1))
v
v @ v.T
np.outer(v, v)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1389975115535016, 'fraction_numerical': 0.018130110202630643, 'mean_word_length': 5.176728869374314, 'pattern_counts': {'":': 0, '<': 65, '<?xml version=': 0, '>': 65, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '70436', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2070, 'n_tokens_neox': 2003, 'n_words': 630, 'path': 'notebooks/S07B_Vectors_Annotated.ipynb', 'repo_name': 'taotangtt/sta-663-2018', 'size': '5626'}