<jupyter_start><jupyter_text>Read in training/testing data<jupyter_code>#Read in train/test data
train_X = pd.read_pickle('Data/train_test/train_X.pkl')
test_X = pd.read_pickle('Data/train_test/test_X.pkl')
train_y = pd.read_pickle('Data/train_test/train_y.pkl')
test_y = pd.read_pickle('Data/train_test/test_y.pkl')<jupyter_output><empty_output><jupyter_text>Random Forest ClassifierWe will start with the Random Forest Classifier which is pretty common becuase of it's predicitive power. We will run using all features first and then we can compare the different amount of features to see what performs best.<jupyter_code>rf = RandomForestClassifier(verbose=True)
with joblib.parallel_backend('dask'):
    rf.fit(train_X, train_y)
rf.score(test_X, test_y)<jupyter_output>[Parallel(n_jobs=-1)]: Using backend DaskDistributedBackend with 6 concurrent workers.
[Parallel(n_jobs=-1)]: Done  38 tasks      | elapsed:   14.4s
[Parallel(n_jobs=-1)]: Done 100 out of 100 | elapsed:   21.3s finished
[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    1.3s finished<jupyter_text>This performs pretty well. WWe can further tune some hyperparamters to possibly get higher accuracy. Let's start with class weights. But we need to first save the parameters and score for later review.<jupyter_code>rf_deets = pd.DataFrame(rf.get_params(), index=['RF_OBO'])
rf_deets.insert(0, value=rf.score(test_X, test_y), column='Accuracy')
rf_deets
rf_class = RandomForestClassifier(class_weight='balanced_subsample',verbose=True)
with joblib.parallel_backend('dask'):
    rf_class.fit(train_X, train_y)
rf_class.score(test_X, test_y)<jupyter_output>[Parallel(n_jobs=-1)]: Using backend DaskDistributedBackend with 6 concurrent workers.
[Parallel(n_jobs=-1)]: Done  38 tasks      | elapsed:   15.1s
[Parallel(n_jobs=-1)]: Done 100 out of 100 | elapsed:   22.8s finished
[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    1.2s finished<jupyter_text>We do see some improvement here, over a tenth of a percentage point. We will keep the class weights as we try different amounts of estimators and the two types of criterion.<jupyter_code>a = rf_class.get_params()
a['Accuracy'] = rf_class.score(test_X, test_y)
rf_deets = pd.concat([rf_deets, pd.DataFrame(a, index=['RF_CLASS'])], axis=0, join='outer')
rf_deets
n_estimators = [50, 100, 250, 500]
criterion = ['gini', 'entropy']
count=1
for c in criterion:
    for n in n_estimators:
        rf_est = RandomForestClassifier(n_estimators=n, criterion=c, class_weight='balanced_subsample', verbose=True)
        with joblib.parallel_backend('dask'):
            rf_est.fit(train_X, np.ravel(train_y))
        deets = rf_est.get_params()
        deets['Accuracy']=rf_est.score(test_X, test_y)
        rf_deets = pd.concat([rf_deets, pd.DataFrame(deets, index=['RF_{n}_{c}'.format(n=n, c=c)])], axis=0, join='outer')
rf_deets
rf_f = RandomForestClassifier(n_estimators=250, criterion='gini', class_weight='balanced_subsample', verbose=True)
with joblib.parallel_backend('dask'):
    rf_f.fit(train_X, np.ravel(train_y))
rf_f.score(test_X, test_y)
y_pred = rf_f.predict(test_X)
cm=confusion_matrix(test_y, y_pred, labels=rf_f.classes_)
disp = ConfusionMatrixDisplay(cm, display_labels=rf_f.classes_)
plt.figure(figsize=(40,8))
disp.plot()
plt.tight_layout()
plt.xticks(rotation=90)
plt.show()
print(classification_report(test_y, y_pred, labels=rf_f.classes_))
print(cross_val_score(rf_f, train_X, train_y, cv=5))
rf_deets.to_pickle('Data/scores/RF.pkl')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12200435729847495, 'fraction_numerical': 0.022331154684095862, 'mean_word_length': 6.375502008032129, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '224481', 'license': 'MIT', 'n_tokens_mistral': 1410, 'n_tokens_neox': 1310, 'n_words': 299, 'path': '3D Modeling-Random Forest.ipynb', 'repo_name': 'Bwils171/Pump-it-up', 'size': '3672'}