<jupyter_start><jupyter_text>This blog will cover four parts:1. *What is random forest*2. *Why we want to use it*3. *How it works *4. *it's implementation in python sklearn*  What is random forest? Random forest is the advanced version of decision tree. It combines/ensembles numbers of decision trees for prediction, instead of just one. Note: If you don't know what decision tree is and are interested in the rationale behind it, please refer my previous blog on decision tree - https://medium.com/bite-sized-machine-learning/decision-tree-classifier-explained-9543dd052746![title](Capture0.PNG)  What's the benefit of using an ensemble method if you are not familiarize with the ensemble method/ensemble learning, you may ask why I need to use it.*Ensemble method*, in general, is a predictive model that make prediction based on a numbers of different models. The top three ensemble architectures are bagging, boosting and stacking. Random forest is an example of bagging, and that will be the focus for this blog.*Bagging* *is using random subsets of the training set to train the model*,  instead of using the entire training set.*Why we need ensemble method?* We can benefit from ensemble method because the individual model usually suffers the bias or variance. By combining different individual model, the ensemble model tends to more flexible (less bias) and less data-sensitive (less variance)* Bias is when the model not paying enough attention to the training data and not flexible enough to adapt along with the training data (e.g. model sticks with some wrong assumption)* Variance is when the model pay too much attention to the training data and captures a lot of noise of the data instead of the pattern  How does random forest work ? *Random forest* is an ensemble model using *bagging *as the ensemble architecture and *decision tree* as individual model as weak learner. *Here are the four steps of building a random forest model:*![title](Capture1.PNG) **Step 1: ** select n (e.g. 1000) random subsets from the training set**Step 2: ** train n decision tree1.  one random subset is used for training one decision tree2. the optimal splits for each decision trees is based on a random subset of features (e.g. 10 feature in total, use a 5 random feature to split)**Step 3:** each tree predicts the records/candidates in the test set, independently. **Step 4:** select the majority vote from these 1000 tree on each record/candidate in the test set as the final decision   Implementation in python sklearn<jupyter_code>from sklearn.datasets import make_moons
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
# Step1: Create data set
X, y = make_moons(n_samples=10000, noise=.5, random_state=0)
# Step2: Split the training test set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Step 3: Fit a Decision Tree model as comparison
clf = DecisionTreeClassifier()
clf.fit(X_train, y_train)

y_pred = clf.predict(X_test)
accuracy_score(y_test, y_pred)
# Step 4: Fit a Random Forest model, " compared to "Decision Tree model, accuracy go up by 5-6%
clf = RandomForestClassifier(n_estimators=100,max_features="auto",random_state=0)
clf.fit(X_train, y_train)

y_pred = clf.predict(X_test)
accuracy_score(y_test, y_pred)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06231884057971015, 'fraction_numerical': 0.01565217391304348, 'mean_word_length': 5.274545454545454, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '409868', 'license': 'Apache-2.0', 'n_tokens_mistral': 965, 'n_tokens_neox': 907, 'n_words': 472, 'path': 'Random Forest/Random Forest - Explained.ipynb', 'repo_name': 'hassanmoid/codes', 'size': '3450'}