<jupyter_start><jupyter_text>Scaling the data before performing PCA<jupyter_code>from sklearn.preprocessing import StandardScaler

features = ['sepal length','sepalwidth','petal length','petal width']

#Separate out features from the target

x = iris.loc[:, features].values

y = iris.loc[:, ['target']].values

# Standardizing the features

print('Before Scaling ',x[0:5,:])

x = StandardScaler().fit_transform(x)

print('After Scaling ',x[0:5,:])<jupyter_output>Before Scaling  [[5.1 3.5 1.4 0.2]
 [4.9 3.  1.4 0.2]
 [4.7 3.2 1.3 0.2]
 [4.6 3.1 1.5 0.2]
 [5.  3.6 1.4 0.2]]
After Scaling  [[-0.90068117  1.03205722 -1.3412724  -1.31297673]
 [-1.14301691 -0.1249576  -1.3412724  -1.31297673]
 [-1.38535265  0.33784833 -1.39813811 -1.31297673]
 [-1.50652052  0.10644536 -1.2844067  -1.31297673]
 [-1.02184904  1.26346019 -1.3412724  -1.31297673]]<jupyter_text>PCA Projection to 2D For Visualization<jupyter_code>from sklearn.decomposition import PCA

pca = PCA(n_components=2)

principal_components = pca.fit_transform(x)

principal_df = pd.DataFrame(data = principal_components, columns = ['principal_component_1','principal_component_2'])

final_df = pd.concat([principal_df, iris['target']], axis = 1)
final_df.head()<jupyter_output><empty_output><jupyter_text>Visualise 2-D projection<jupyter_code>import matplotlib.pyplot as plt

fig = plt.figure(figsize=(8,8))
ax = fig.add_subplot(1,1,1)
ax.set_xlabel('Principal Component 1', fontsize = 15)
ax.set_ylabel('Principal Component 2', fontsize = 15)
ax.set_title('2 component PCA', fontsize = 20)

targets = ['Iris-setosa', 'Iris-versicolor', 'Iris-virginica']
colors = ['r', 'g', 'b']

for target,color in zip(targets,colors):
    indices_to_keep = final_df['target'] == target
    ax.scatter(final_df.loc[indices_to_keep, 'principal_component_1'], final_df.loc[indices_to_keep, 'principal_component_2']
    , c = color, s = 50)
ax.legend(targets)
ax.grid()<jupyter_output><empty_output><jupyter_text>As we know that PCA is a variance maximization algorithm with a goal to minimise the sum of squared difference between the projected datapoints and the original ones. Take a look at the explained variance of each principal component to visualise how much variance is captured by them:<jupyter_code>pca.explained_variance_ratio_<jupyter_output><empty_output><jupyter_text>Thus first principal component explains 72.7 % of variance whereas the second component explains 23.03 of the variance, so in total our PCA transformation explains around 95% of the variance of original dimensions.<jupyter_code>from sklearn.datasets import fetch_mldata
mnist = fetch_mldata('MNIST original',data_home='D:/upgrad/DimensionalityReduction/mnist')
type(mnist.data)
# Split into training and test dataset
from sklearn.model_selection import train_test_split

train_img, test_img, train_lbl, test_lbl = train_test_split(mnist.data, mnist.target, test_size=1/7.0, random_state=0)
train_img.shape
test_img.shape
train_lbl.shape
test_lbl.shape
# Apply Standardisation on the data

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

# Fit for the training data only
scaler.fit(train_img)

# Apply the transform to both the training and test dataset

train_img = scaler.transform(train_img)

test_img = scaler.transform(test_img)<jupyter_output>D:\ProgramData\Anaconda3\lib\site-packages\sklearn\utils\validation.py:475: DataConversionWarning: Data with input dtype uint8 was converted to float64 by StandardScaler.
  warnings.warn(msg, DataConversionWarning)<jupyter_text>Now that the data is standardized next step is to apply the PCA tranformation<jupyter_code>from sklearn.decomposition import PCA

pca = PCA(0.95)

# fit the PCA transformation on the training set

pca.fit(train_img)
# transforming the train and test dataset with the PCA transformation

train_img = pca.transform(train_img)
test_img = pca.transform(test_img)
print(train_img.shape)
print(test_img.shape)<jupyter_output>(10000, 330)<jupyter_text>Apply Logistic Regression on The Transformed Data<jupyter_code>from sklearn.linear_model import LogisticRegression

logist_regr = LogisticRegression(solver='lbfgs')
logist_regr.fit(train_img,train_lbl)
# Predict For one observation

logist_regr.predict(train_img[0].reshape(1,-1))
logist_regr.score(test_img, test_lbl)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10823311748381129, 'fraction_numerical': 0.06359851988899168, 'mean_word_length': 6.196339434276206, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '624284', 'license': 'MIT', 'n_tokens_mistral': 1682, 'n_tokens_neox': 1522, 'n_words': 368, 'path': 'PCA_Iris.ipynb', 'repo_name': 'aryancodify/DimensionalityReducton', 'size': '4324'}