<jupyter_start><jupyter_text>Importing Packages<jupyter_code>import pandas as pd
import numpy as np
import seaborn as sns
#For visualising the screeplot
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Linear Combination<jupyter_code>v1 = np.array([2,3])
v2 = np.array([3,4])
m = 1
n = 2

lincom = m*v1 +  n*v2
lincom<jupyter_output><empty_output><jupyter_text>We can do the reverse part as well. That is finding the values of m,n for which the given vector is a linear combination of the previous two vectors.​<jupyter_code>lincom2  = ([8,11])
setvec = np.array([[2,3],[3,4]])
m,n = np.linalg.solve(setvec, lincom2)
m,n<jupyter_output><empty_output><jupyter_text>Basis Vector<jupyter_code>lincom2  = ([6,3])
setvec = np.array([[2,3],[3,4]])
m,n = np.linalg.solve(setvec, lincom2)
round(m),round(n)<jupyter_output><empty_output><jupyter_text>What is the issue below? Why I am not able to represent (6,3) in given set vectors?<jupyter_code>lincom2  = ([6,3])
setvec = np.array([[2,0],[3,0]])
m,n = np.linalg.solve(setvec, lincom2)
round(m),round(n)<jupyter_output><empty_output><jupyter_text>Properties of standard coordinate system​<jupyter_code># The vectors are orthogonal.
setvec = np.array([[1,0],[0,1]])
np.dot(setvec[0],setvec[1])
##This would mean that each of them would be of unit magnitude
print(np.linalg.norm(setvec[0]))
print(np.linalg.norm(setvec[1]))<jupyter_output>1.0
1.0<jupyter_text>Linear Transformation  Elongation<jupyter_code># Data Frame where each row present a datapoint
a = [[1,2],[-2,3],[-2,1],[3,7],[4,5],[6,4]]
b = ['X','Y']
c = pd.DataFrame(a,columns = b)
c
# Linear transformation matrix
L = np.array([[2,0],[0,1]])
L
#Let's apply the transformation now.
d  = L @ (c.values).T
d.T<jupyter_output><empty_output><jupyter_text>Shifting of axes<jupyter_code>c
L = np.array([[0.8,-0.6],[0.6,0.8]])

#Finding the inverse and then using the transformation on the given set of points.
Ld = np.linalg.inv(L)

a = [[1,2],[-2,3],[-2,1],[3,7],[4,5],[6,4]]
b = ['X','Y']
c = pd.DataFrame(a,columns = b)

d  = Ld @ (c.values).T
d.T<jupyter_output><empty_output><jupyter_text>Basis Transformation<jupyter_code>a=[[1,2],[2,4],[3,6],[4,8],[5,10],[6,12],[7,14]]
b=['X','Y']
c = pd.DataFrame(a,columns = b)
c
#Let's visualise to notice if any pattern exists
sns.scatterplot(x ='X', y='Y',data=c)
# Let's represent the datapoint int he polar coordinate p
p = np.array([[0.4472136,-0.89442719],[ 0.89442719,0.4472136]])
p
pI= np.linalg.inv(p)

# Performing the basis transformation.
C= pI @ c.T
C.T
D = C.T
D.columns = ['X', 'Y']
D['Y']=abs(round(D['Y']))
D
#Let's plot the scatterplot again.
sns.scatterplot(x='X',y='Y',data =D)<jupyter_output><empty_output><jupyter_text>Now you can see that all the data points are aligned along the new X axis and have no values on the new Y axis. This is essentially what you achieve by doing basis transformation.  Eigenvalues and Eigenvectors<jupyter_code>A = np.array([[2,1],[1,2]])

K = np.linalg.eig(A)

#Eigenvalues
print('Eigenvalues', K[0])

#Eigenvectors
print('Eigenvectors', K[1])

# Verify the results
# Let's verify them once again. First, let's calculate Av
print('Av',A @ (K[1].T)[0])

#Now let's calculate λv.
print('λv',3*(K[1].T)[0])

# Output:
# As you can see Av = λv has been verified.<jupyter_output>Eigenvalues [3. 1.]
Eigenvectors [[ 0.70710678 -0.70710678]
 [ 0.70710678  0.70710678]]
Av [2.12132034 2.12132034]
λv [2.12132034 2.12132034]<jupyter_text>Intresenting Calculation<jupyter_code># Arrange the eigenvalues along the diagonal of a square matrix.
S = np.array([[3,0], [0,1]])

# Take eigenvector
Q=K[1]

# take the inverse of eigenvector
P = np.linalg.inv(Q)

# The product of the three matrices obtained in the following order.
Q @ S @ P<jupyter_output><empty_output><jupyter_text>From eigenvectors and eigenvalue we can obtain the original matrix from where we have started.​  PCA Properties<jupyter_code>#Let's now take the following dataframe
a = [[0,0,0],[1,2,3],[2,4,5],[3,6,2],[4,8,3],[5,10,1]]
b = ['X','Y','Z']
dat = pd.DataFrame(a,columns = b)
dat<jupyter_output><empty_output><jupyter_text>PCA Properties- They're weighted linear combinations of the original variables.- They're perpendicular to each other.- They capture maximum variance of the data and are ordered.<jupyter_code>#Importing the PCA module
from sklearn.decomposition import PCA
pca = PCA(svd_solver='randomized', random_state=42)
pca.fit(dat)
pca.components_<jupyter_output><empty_output><jupyter_text>They're weighted linear combinations of the original variables.<jupyter_code># This value can be represented as a linear combination of the original variables. 
# The original variables or the original axes are represented by X(1,0,0) , Y(0,1,0) and Z(0,0,1).
# [-0.45,-0.89,-0.03] = -0.45[1,0,0] + (-0.89)[0,1,0] + (-0.03)[0,0,1]
#     = -0.45X + (-0.89)Y +(-0.03)Z,  which is a linear combination of the original variables
lincom  = pca.components_[0]
setvec = np.array([[1,0,0],[0,1,0],[0,0,1]])
x,y,z = np.linalg.solve(setvec, lincom)
round(x,2),round(y,2),round(z,2)<jupyter_output><empty_output><jupyter_text>They're perpendicular to each other.<jupyter_code>round(np.dot(pca.components_[0],pca.components_[1]))<jupyter_output><empty_output><jupyter_text>They capture maximum variance of the data and are ordered.<jupyter_code>pca.explained_variance_ratio_.round(2)
pca.fit_transform(dat)
## Projecting datapoints on Principal Components
#We can directly use the pca function pca.fit_transform
#This is equivalent to the basis transformation that we did in last session.

dat_new2 = pca.fit_transform(dat)
cols = ['X','Y','Z']
dat_new3 = pd.DataFrame(dat_new2, columns = cols).round(2)
dat_new3
100*np.var(dat_new3)/sum(np.var(dat_new3))<jupyter_output><empty_output><jupyter_text>Let's do a final check. Lets see the varince of original data and transformed data<jupyter_code>sum(np.var(dat)), sum(np.var(dat_new3))<jupyter_output><empty_output><jupyter_text>This is what PCA does. It doesn't change the total variance of the dataset. It only rearranges them in the direction of maximum variances  PCA using Eigendecomposition<jupyter_code>#Let's take this dataset
a = [[0,0],[1,2],[2,3],[3,6],[4,8],[5,9]]
b = ['X','Y']
dat = pd.DataFrame(a,columns = b)
dat
# A reason as to why we're doing this is to capture the variance of the entire dataset
C = np.cov(dat.T)
C
eigenvalues, eigenvectors = np.linalg.eig(C)
# Sorting eigenvectors based on eigen value
idx = eigenvalues.argsort()[::-1]   
eigenvalues= eigenvalues[idx]
eigenvectors = eigenvectors[:,idx]<jupyter_output><empty_output><jupyter_text>Let's look at the data points in PC components<jupyter_code>Cnew = np.linalg.inv(eigenvectors) @ dat.T
cols  = ['X','Y']
Cnewt = Cnew.T
Cnewt.columns = cols
Cnewt
100*np.var(Cnewt)/sum(np.var(Cnewt))
eigenvectors<jupyter_output><empty_output><jupyter_text>Let's compare it to the PCA<jupyter_code>pca = PCA(svd_solver='randomized', random_state=42)
pca.fit(dat)
pca.components_
pca.explained_variance_ratio_<jupyter_output><empty_output><jupyter_text>Why? Spectral Theorem. Because of this theorem eigendecomposition of the covariance matrix will always:1. Yield the eigenvectors which are perpendicular to each other 2. Have maximum variances allocated to them in an ordered way depending on the magnitude of the eigenvalues.3. These eigenvectors would also be the linear combinations of the original variables as well.  Using SVD Function<jupyter_code>#Let'se use the SVD function from np.linalg library
U, s, VT = np.linalg.svd(dat, full_matrices=False)
U
s
VT
eigenvectors
# Project the data now by doing the basis transformation
datn = np.linalg.inv(eigenvectors) @ dat.T
datn.T
# convert s matrix to a diagonal one
o = U @ np.array([[15.77196179,0],[0,0.49519836]])
o<jupyter_output><empty_output><jupyter_text>U.s = Representing the datapoints in the Principal components - SVD overcomes the main problem of eigendecomposition where the matrix needs to be square and diagonalizable.- SVD enables us to decompose the data to find hidden themes in a more succint way  Screeplots<jupyter_code>%matplotlib inline
fig = plt.figure(figsize = (12,8))
plt.plot(np.cumsum(pca.explained_variance_ratio_))
plt.xlabel('number of components')
plt.ylabel('cumulative explained variance')
plt.show()
ratings = pd.read_csv("Ratings/Book-Ratings.csv")
pca = PCA(svd_solver='randomized', random_state=42)
pca.fit(ratings)
%matplotlib inline
fig = plt.figure(figsize = (12,8))
plt.plot(np.cumsum(pca.explained_variance_ratio_))
plt.xlabel('number of components')
plt.ylabel('cumulative explained variance')
plt.show()
# 3 principal components are sufficient in explaining 98 % of the variance.<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14302046415914027, 'fraction_numerical': 0.042528867040128045, 'mean_word_length': 6.2960800667222685, 'pattern_counts': {'":': 0, '<': 87, '<?xml version=': 0, '>': 87, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '465660', 'license': 'MIT', 'n_tokens_mistral': 3511, 'n_tokens_neox': 3289, 'n_words': 843, 'path': 'Unsupervised Learning/PCA/PCA Explanation.ipynb', 'repo_name': 'tavishaggarwal/Data-Science-Learning', 'size': '8747'}