<jupyter_start><jupyter_text>1. Scikit learn comes with pre-loaded dataset, load the digits dataset from that collection and write a helper function to plot the image using matplotlib.<jupyter_code>digits = load_digits()
plt.gray()
plt.matshow(digits.images[0])
plt.show()
plt.gray()
plt.matshow(digits.images[2])
plt.show()<jupyter_output><empty_output><jupyter_text>2. Make a train -test split with 20% of the data set aside for testing. Fit a logistic regression model and observe the accuracy.<jupyter_code>x = digits.data
y = digits.target

x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.2,random_state=2)
print(x_train.shape)
print(y_train.shape)
print(x_test.shape)
print(y_test.shape)
logistic = LogisticRegression()
logistic = logistic.fit(x_train,y_train)
y_predict = logistic.predict(x_test)

# accuracy:
accuracy_score(y_test,y_predict)
confusion_matrix(y_test,y_predict)
print(classification_report(y_test,y_predict))<jupyter_output>precision    recall  f1-score   support

           0       1.00      0.97      0.98        32
           1       0.95      0.93      0.94        44
           2       1.00      1.00      1.00        31
           3       0.94      0.92      0.93        36
           4       0.94      0.89      0.91        35
           5       0.98      0.95      0.96        43
           6       1.00      0.94      0.97        35
           7       0.97      0.97      0.97        40
           8       0.81      0.97      0.89        36
           9       0.86      0.89      0.88        28

    accuracy                           0.94       360
   macro avg       0.95      0.94      0.94       360
weighted avg       0.95      0.94      0.95       360<jupyter_text>3. Using scikit learn perform a PCA transformation such that the transformed dataset can explain 95% of the variance in the original dataset. Find out the number of components in the projected subspace<jupyter_code>model_pca = PCA(n_components=29)

model_pca.fit(images)

plt.plot(np.cumsum(model_pca.explained_variance_ratio_))
print(np.cumsum(model_pca.explained_variance_ratio_))
plt.show()
# 29 principal components explain almost 95% of variance in the dataset.<jupyter_output><empty_output><jupyter_text>4. Transform the dataset and fit a logistic regression and observe the accuracy.Compare it with the previous model and comment on the accuracy.<jupyter_code>train_x = model_pca.transform(x_train)
test_x = model_pca.transform(x_test)
logistic.fit(train_x, y_train)

predicted_values_1 = logistic.predict(test_x)

print("Variance Ratio\n")
accuracy_score(predicted_values_1, y_test)
# the accuracy with 29 variables is 93.6 % compared to accuracy of 94% with 64 variables. The dimensionality reduction has same
# impact.<jupyter_output><empty_output><jupyter_text>5. Compute the confusion matrix and count the number of instances that has gone wrong. For each of the wrong sample, plot the digit along with predicted and original label.<jupyter_code>confusion_matrix(y_test,predicted_values_1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07757774140752864, 'fraction_numerical': 0.05990180032733224, 'mean_word_length': 2.7496932515337424, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '468881', 'license': 'Apache-2.0', 'n_tokens_mistral': 1083, 'n_tokens_neox': 981, 'n_words': 276, 'path': 'Module8/Module_8-Case+study+1.ipynb', 'repo_name': 'Hemanthvijaykumar/Edurekha_Python-For-Datascience', 'size': '3055'}