<jupyter_start><jupyter_text>Fetching and Splitting Cleaned Data into Test and Train Sets<jupyter_code>flairs = flairs= ["Scheduled","Politics","Photography","Policy/Economy","AskIndia","Sports",
         "Non-Political","Science/Technology","Food","Business/Finance","Coronavirus"]
data = pd.read_csv('../data/cleansedData300.csv')
data["combined"] = data.title.astype("str")+" "+ data.url.astype("str")+" "+ data.author.astype("str")+" "+data.authors.astype("str")+" "+ data.comments.astype("str")
y = data.flair
X = data.combined
X_train, X_test, y_train, y_test = train_test_split(X, y,test_size=0.1,random_state = 5)<jupyter_output><empty_output><jupyter_text>Performing Machine Learning Algorithms<jupyter_code>accNB = multinomialNaiveBayes(X_train, X_test, y_train, y_test)
print("Multi-Nomial Naive Bayes Accuracy: ",accuracy_score(accNB, y_test))
print()
print(classification_report(y_test, accNB,target_names=flairs))
accLR = logisticRegression(X_train, X_test, y_train, y_test)
print("Logistic Regression Accuracy: ",accuracy_score(accLR, y_test))
print()
print(classification_report(y_test, accLR,target_names=flairs))
accRF = randomForest(X_train, X_test, y_train, y_test)
print("Random Forest Accuracy: ",accuracy_score(accRF, y_test))
print()
print(classification_report(y_test, accRF,target_names=flairs))
accMLP = multiLayerPerceptron(X_train, X_test, y_train, y_test)
print("Multi Layer Perceptron Accuracy: ",accuracy_score(accMLP, y_test))
print()
print(classification_report(y_test, accMLP,target_names=flairs))
accSVM = mySVM(X_train, X_test, y_train, y_test)
print("SVM Accuracy: ",accuracy_score(accSVM, y_test))
print()
print(classification_report(y_test, accMLP,target_names=flairs))<jupyter_output>SVM Accuracy:  0.6367041198501873

                    precision    recall  f1-score   support

         Scheduled       0.38      0.21      0.27        28
          Politics       0.64      0.24      0.35        29
       Photography       0.68      0.53      0.60        32
    Policy/Economy       0.46      0.68      0.55        19
          AskIndia       0.26      0.29      0.27        24
            Sports       0.81      0.61      0.69        28
     Non-Political       0.26      0.71      0.37        17
Science/Technology       0.61      0.46      0.52        24
              Food       1.00      1.00      1.00        27
  Business/Finance       0.35      0.52      0.42        25
       Coronavirus       0.90      0.64      0.75        14

          accuracy                           0.52       267
         macro avg       0.58      0.54      0.53       267
      weighted avg       0.58      0.52      0.53       267<jupyter_text>Saving Naive Bayes Model since it was the best of all<jupyter_code>nb = Pipeline([('vect', CountVectorizer()),
           ('tfidf', TfidfTransformer()),
           ('clf', MultinomialNB()),
          ])

tempNB = nb.fit(X_train, y_train)
pickle.dump(tempNB,open("../data/finalModel.sav","wb"))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12008005336891261, 'fraction_numerical': 0.058372248165443626, 'mean_word_length': 2.7628607277289836, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '230581', 'license': 'Apache-2.0', 'n_tokens_mistral': 1152, 'n_tokens_neox': 1030, 'n_words': 196, 'path': 'code/.ipynb_checkpoints/machineLearning-checkpoint.ipynb', 'repo_name': 'xrachitx/Reddit-Flair-Detector', 'size': '2998'}