<jupyter_start><jupyter_text>Singular Value DecompositionGeneralization of eigen decomposition to non-square matrices. SVD:  * $A = U \Sigma V^*$  * U and V are unitary. (For real numbers in A, this means U and V is each orthogonal and $V^*$=$V^T$. For complex A, unitary means the complex conjugate equals the inverse for U and V.)* A = m x n, U = m x m, $\Sigma$ = m x n, V = n x n.* $\Sigma$ is diagonal and its non-zero diagonal elements are the singular values.* Number of singular values = rank(A).* There is one and only one SVD having singular values in descending order.* The columns of U and V are orthonormal bases (orthogonal, unit length). Relationship to eigen decomposition:* SVD generalizes to non-square matrices (eigen for square only)* SVD uses $A = U \Sigma V^T$ (eigen uses $A = U \Sigma U^T$* For $A=B^TB$ or $A=BB^T$, singular values of B are the square roots of the eigen values of A.* For symmetric matrix A, SVD = eigen (I think). Facts:* SVD is used to compute pseudo-inverse of a matrix.* SVD is used to compute linear least squares (requires pseudo-inverse). * SVD is used for Tikhinov regularization (and the special case Ridge regularization).* rank(A) = number of non-zero singular values.* Retain only large singular values to achieve low-rank approximation of matrix A.<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>SVD with numpy Use this example.Same example was used for Eigen.  https://www.d.umn.edu/~mhampton/m4326svd_example.pdf  Square symmetric matrix<jupyter_code># 2 rows, 3 columns
B = np.array([[3,2,2],[2,3,-2]])
A = B.dot(B.T)
print("Square matrix A = BBT\n",A)
U,Sigma,Vt = np.linalg.svd(A)
print("U\n",U);
print("Sigma\n",Sigma);
print("V transpose\n",Vt);<jupyter_output>U
 [[-0.70710678 -0.70710678]
 [-0.70710678  0.70710678]]
Sigma
 [25.  9.]
V transpose
 [[-0.70710678 -0.70710678]
 [-0.70710678  0.70710678]]<jupyter_text>Non-square matrix<jupyter_code>U,Sigma,Vt = np.linalg.svd(B)
print("U\n",U);
print("Singular values in descending order\n",Sigma);
print("V transpose\n",Vt);<jupyter_output>U
 [[-0.70710678 -0.70710678]
 [-0.70710678  0.70710678]]
Sigma
 [5. 3.]
V transpose
 [[-7.07106781e-01 -7.07106781e-01 -6.47932334e-17]
 [-2.35702260e-01  2.35702260e-01 -9.42809042e-01]
 [-6.66666667e-01  6.66666667e-01  3.33333333e-01]]<jupyter_text>Round-trip validation. Use B=U*S*V.<jupyter_code>S = [[5,0,0],[0,3,0]]  
print("Round trip validation:\n",U.dot(S).dot(Vt))<jupyter_output>Round trip validation:
 [[ 3.  2.  2.]
 [ 2.  3. -2.]]
{'fraction_non_alphanumeric': 0.14370546318289787, 'fraction_numerical': 0.0934283452098179, 'mean_word_length': 4.434408602150538, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 3, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '464819', 'license': 'MIT', 'n_tokens_mistral': 1072, 'n_tokens_neox': 927, 'n_words': 299, 'path': 'PR677/Singular_Value_Decomposition.ipynb', 'repo_name': 'ShepherdCode/ShepherdML', 'size': '2526'}