<jupyter_start><jupyter_text>2.3.1. Scalars<jupyter_code>x = np.array(3.0)
y = np.array(2.0)

x + y, x * y, x / y, x ** y<jupyter_output><empty_output><jupyter_text>2.3.2. Vectors<jupyter_code>x = np.arange(4)
x
x[3]<jupyter_output><empty_output><jupyter_text>**[NOTE]** Note that the word “dimension” tends to get overloaded in these contexts and this tends to confuse people. To clarify, we use the dimensionality of a vector or an axis to refer to its length, i.e., the number of elements of a vector or an axis. However, we use the dimensionality of an ndarray to refer to the number of axes that an ndarray has. In this sense, the dimensionality of some axis of an ndarray will be the length of that axis.<jupyter_code>len(x), x.shape<jupyter_output><empty_output><jupyter_text>2.3.3. Matrices<jupyter_code>A = np.arange(20).reshape(5, 4)
A
A[1], A[1][1]<jupyter_output><empty_output><jupyter_text>transpose of the matrix<jupyter_code>A.T
B = np.array([[1, 2, 3], [2, 0, 4], [3, 4, 5]])
B
B == B.T<jupyter_output><empty_output><jupyter_text>2.3.4. Tensors<jupyter_code>X = np.arange(24).reshape(2, 3, 4)
X<jupyter_output><empty_output><jupyter_text>2.3.5. Basic Properties of Tensor Arithmetic<jupyter_code>A = np.arange(20).reshape(5, 4)
B = A.copy()  # Assign a copy of A to B by allocating new memory
A, A + B<jupyter_output><empty_output><jupyter_text>**elementwise multiplication of two matrices is called their Hadamard product (math notation  ⊙ )**<jupyter_code>A * B
a = 2
X = np.arange(24).reshape(2, 3, 4)
a + X, (a * X).shape<jupyter_output><empty_output><jupyter_text>2.3.6. Reduction<jupyter_code>x = np.arange(4)
x, x.sum()
A
A.shape, A.sum()
A_sum_axis0 = A.sum(axis=0)
A_sum_axis0, A_sum_axis0.shape
A_sum_axis1 = A.sum(axis=1)
A_sum_axis1, A_sum_axis1.shape
A.sum(axis=[0, 1])  # Same as A.sum()
A.mean(), A.sum() / A.size
A.mean(axis=0), A.sum(axis=0) / A.shape[0]<jupyter_output><empty_output><jupyter_text>2.3.6.1. Non-Reduction Sum<jupyter_code>sum_A = A.sum(axis=1, keepdims=True)
sum_A, sum_A.shape
A / sum_A
A.cumsum(axis=0)<jupyter_output><empty_output><jupyter_text>2.3.7. Dot Products<jupyter_code>y = np.ones(4)
x, y, np.dot(x, y)
(x * y).sum()<jupyter_output><empty_output><jupyter_text>2.3.8. Matrix-Vector Products<jupyter_code>A, x
(A[0] * x).sum()
A.shape, x.shape, np.dot(A, x)
A*x, np.dot(A, x)<jupyter_output><empty_output><jupyter_text>2.3.9. Matrix-Matrix Multiplication<jupyter_code>B = np.ones(shape=(4, 3))
np.dot(A, B)<jupyter_output><empty_output><jupyter_text>2.3.10. Norms > Informally, the norm of a vector tells us how big a vector is. The notion of size under consideration here concerns not dimensionality but rather the magnitude of the components. The  ℓ2  norm of  𝐱  is the square root of the sum of the squares of the vector elements:$$\|\mathbf{x}\|_2 = \sqrt{\sum_{i=1}^n x_i^2},$$<jupyter_code>u = np.array([3, -4])
np.linalg.norm(u)<jupyter_output><empty_output><jupyter_text>the  ℓ1  norm, which is expressed as the sum of the absolute values of the vector elements:$$\|\mathbf{x}\|_1 = \sum_{i=1}^n \left|x_i \right|.$$<jupyter_code>np.abs(u).sum()<jupyter_output><empty_output><jupyter_text>Both the $\ell_2$ norm and the $\ell_1$ normare special cases of the more general $\ell_p$ *norm*:$$\|\mathbf{x}\|_p = \left(\sum_{i=1}^n \left|x_i \right|^p \right)^{1/p}.$$Analogous to $\ell_2$ norms of vectors,the *Frobenius norm* of a matrix $\mathbf{X} \in \mathbb{R}^{m \times n}$is the square root of the sum of the squares of the matrix elements:$$\|\mathbf{X}\|_F = \sqrt{\sum_{i=1}^m \sum_{j=1}^n x_{ij}^2}.$$The Frobenius norm satisfies all the properties of vector norms.It behaves as if it were an $\ell_2$ norm of a matrix-shaped vector. Invoking `linalg.norm` will calculate the Frobenius norm of a matrix.<jupyter_code>np.linalg.norm(np.ones((4, 9)))<jupyter_output><empty_output><jupyter_text>2.3.10.1. Norms and Objectives While we do not want to get too far ahead of ourselves, we can plant some intuition already about why these concepts are useful. In deep learning, we are often trying to solve optimization problems: maximize the probability assigned to observed data; minimize the distance between predictions and the ground-truth observations. Assign vector representations to items (like words, products, or news articles) such that the distance between similar items is minimized, and the distance between dissimilar items is maximized. Oftentimes, the objectives, perhaps the most important components of deep learning algorithms (besides the data), are expressed as norms.  2.3.13. Exercises 1. Prove that the transpose of a matrix  𝐀 ’s transpose is  𝐀 :  (𝐀⊤)⊤=𝐀 .<jupyter_code>A
(A.T).T == A<jupyter_output><empty_output><jupyter_text>2. Given two matrices  𝐀  and  𝐁 , show that the sum of transposes is equal to the transpose of a sum:  𝐀⊤+𝐁⊤=(𝐀+𝐁)⊤ .<jupyter_code>B = A.copy()
A, B
A.T + B.T  == (A + B).T<jupyter_output><empty_output><jupyter_text>4. We defined the tensor X of shape ( 2 ,  3 ,  4 ) in this section. What is the output of len(X)?<jupyter_code>X
len(X)<jupyter_output><empty_output><jupyter_text>6. Run A / A.sum(axis=1) and see what happens. Can you analyze the reason?<jupyter_code>try:
    print(A / A.sum(axis=1))
except Exception as e:
    print(f"[Error] {e}")<jupyter_output>[Error] Incompatible shapes for broadcasting: ((5, 4), (1, 5))<jupyter_text>8. Consider a tensor with shape ( 2 ,  3 ,  4 ). What are the shapes of the summation outputs along axis  0 ,  1 , and  2 ?<jupyter_code>X.shape
X.sum(axis=0), X.sum(axis=0).shape
X.sum(axis=1), X.sum(axis=1).shape
X.sum(axis=2), X.sum(axis=2).shape<jupyter_output><empty_output><jupyter_text>9. Feed a tensor with 3 or more axes to the linalg.norm function and observe its output. What does this function compute for ndarrays of arbitrary shape?<jupyter_code>np.linalg.norm(np.ones((4, 9, 10, 20)))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13975576662143827, 'fraction_numerical': 0.02628900949796472, 'mean_word_length': 5.3002136752136755, 'pattern_counts': {'":': 0, '<': 88, '<?xml version=': 0, '>': 89, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '657483', 'license': 'MIT', 'n_tokens_mistral': 2371, 'n_tokens_neox': 2291, 'n_words': 648, 'path': '02-preliminaries/2.3-linear-algebra.ipynb', 'repo_name': 'AmrMKayid/d2l-jax', 'size': '5896'}