<jupyter_start><jupyter_text>7. Transform the labels - (7.5 points)- As we have noticed before, in this task each example can have multiple tags. To deal withsuch kind of prediction, we need to transform labels in a binary form and the prediction will bea mask of 0s and 1s. For this purpose, it is convenient to use MultiLabelBinarizer from sklearn<jupyter_code>#### a. Convert your train and test labels using MultiLabelBinarizer

mlb = MultiLabelBinarizer(classes = sorted(label_counts.keys()))

y_train_mlb = mlb.fit_transform(y_train)

y_test_mlb = mlb.fit_transform(y_test)


mlb.classes_

y_train_mlb[0:2]

y_test_mlb[0:2]
## 8. Choose a classifier - (5 points)

# In this task, we suggest using the One-vs-Rest approach, which is implemented in OneVsRestClassifier class. In this approach k classifiers (= number of tags) are trained. As a
# basic classifier, use LogisticRegression . It is one of the simplest methods, but often it performs good enough in text classification tasks. It might take some time because the
# number of classifiers to train is large.
#### a. Use a linear classifier of your choice, wrap it up in OneVsRestClassifier to train it on every label

logreg_clf = LogisticRegression(solver='lbfgs', max_iter=1000, random_state=42)
#### b. One-vs-Rest approach 

OnevsRest_clf = OneVsRestClassifier(estimator=logreg_clf)
### 9. Fit the classifier, make predictions and get the accuracy (5 points)

#Fit the Classifier on on count vectors for Training dataset

OnevsRest_clf.fit(X_bow_train, y_train_mlb)
#classifier's score for training data

OnevsRest_clf.score(X_bow_train, y_train_mlb)
#classes present in the multiclass classification

OnevsRest_clf.classes_
#Predicted values on count vectors for Test dataset

y_pred = OnevsRest_clf.predict(X_bow_test)

y_pred<jupyter_output><empty_output><jupyter_text>a. Print the following- i. Accuracy score- ii. F1 score- iii. Average precision score- iv. Average recall score-  v. Tip: Make sure you are familiar with all of them. How would you expect thethings to work for the multi-label scenario? Read about micro/macro/weightedaveraging- Three metrics are aavilable for evaluation in multiclass classification - macro, micro and weigted.Here, we are using weighted as the parameter"macro" simply calculates the mean of the binary metrics, giving equal weight to each class. In problems where infrequent classes are nonetheless important, macro-averaging may be a means of highlighting their performance. On the other hand, the assumption that all classes are equally important is often untrue, such that macro-averaging will over-emphasize the typically low performance on an infrequent class."weighted" accounts for class imbalance by computing the average of binary metrics in which each class’s score is weighted by its presence in the true data sample."micro" gives each sample-class pair an equal contribution to the overall metric (except as a result of sample-weight). Rather than summing the metric per class, this sums the dividends and divisors that make up the per-class metrics to calculate an overall quotient. Micro-averaging may be preferred in multilabel settings, including multiclass classification where a majority class is to be ignored.<jupyter_code>#Multilabel Confusion Matrix

print(multilabel_confusion_matrix(y_test_mlb, y_pred))
#Classification Report

report = classification_report(y_test_mlb, y_pred, output_dict=True, zero_division=1)
#DataFrame to store Classification Report

cr_df = pd.DataFrame(report).transpose()

cr_df.head(10)
#Accuracy score

print(accuracy_score(y_test_mlb, y_pred))

#F1 score

print(f1_score(y_test_mlb, y_pred, average='weighted'))

#Precision score

print(precision_score(y_test_mlb, y_pred, average='weighted', zero_division=1))
# Recall score

print(recall_score(y_test_mlb, y_pred, average='weighted'))
#Average Precision score

print(average_precision_score(y_test_mlb, y_pred, average='weighted'))
#Average Recall score
#support is the number of samples of the true positives that lie in that class.
#avearge of the recall for each calss for the suport for the class weighted by total support for all classes

sum((cr_df.iloc[0:80]['recall']) * (cr_df.iloc[0:80]['support']))/ sum(cr_df.iloc[0:80]['support'])
#ROC-AUC score

print(roc_auc_score(y_test_mlb, y_pred, average='weighted'))
### 10. Print true label and predicted label for any five examples (7.5 points)

##### True labels for five examples

y_test_mlb


#Inverse Transform multiclass masked labels to true labels

y_test_labels = mlb.inverse_transform(y_test_mlb)

y_test_labels[0:5]
y_test[0:5]
##### Predicted labels for five examples

y_pred
#Predicted Probabilities for test dataset

proba = OnevsRest_clf.predict_proba(X_bow_test)

proba[0]
#Inverse Transform multiclass probabilities to predicted labels usingig the probabilities above 0.5 threshold

bitarray = OnevsRest_clf.label_binarizer_.inverse_transform(proba, threshold=0.5)
classnames = mlb.inverse_transform(bitarray)

classnames[0:6]
#Inverse Transform multiclass masked labels to predicted labels

y_pred_labels = mlb.inverse_transform(y_pred)

y_pred_labels[0:6]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07097273254689615, 'fraction_numerical': 0.010056081995745503, 'mean_word_length': 5.923694779116466, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '509961', 'license': 'MIT', 'n_tokens_mistral': 1616, 'n_tokens_neox': 1523, 'n_words': 592, 'path': 'Blog_Authorship_NLP.ipynb', 'repo_name': 'swaroop9ai9/Learning-development', 'size': '5171'}