<jupyter_start><jupyter_text>*This tutorial is part of the series [Learn Machine Learning](https://www.kaggle.com/learn/machine-learning). At the end of this step, you will be able to use your first sophisticated machine learning model, the Random Forest.* IntroductionDecision trees leave you with a difficult decision. A deep tree with lots of leaves will overfit because each prediction is coming from historical data from only the few houses at its leaf. But a shallow tree with few leaves will perform poorly because it fails to capture as many distinctions in the raw data.Even today's most sophisticated modeling techniques face this tension between underfitting and overfitting. But, many models have clever ideas that can lead to better performance. We'll look at the **random forest** as an example.The random forest uses many trees, and it makes a prediction by averaging the predictions of each component tree. It generally has much better predictive accuracy than a single decision tree and it works well with default parameters. If you keep modeling, you can learn more models with even better performance, but many of those are sensitive to getting the right parameters.  ExampleYou've already seen the code to load the data a few times. At the end of data-loading, we have the following variables:- train_X- val_X- train_y- val_y<jupyter_code>import pandas as pd
    
# Load data
melbourne_file_path = './data/melbourne-housing-snapshot/melb_data.csv'
melbourne_data = pd.read_csv(melbourne_file_path) 
# Filter rows with missing values
melbourne_data = melbourne_data.dropna(axis=0)
# Choose target and predictors
y = melbourne_data.Price
melbourne_predictors = ['Rooms', 'Bathroom', 'Landsize', 'BuildingArea', 
                        'YearBuilt', 'Lattitude', 'Longtitude']
X = melbourne_data[melbourne_predictors]

from sklearn.model_selection import train_test_split

# split data into training and validation data, for both predictors and target
# The split is based on a random number generator. Supplying a numeric value to
# the random_state argument guarantees we get the same split every time we
# run this script.
train_X, val_X, train_y, val_y = train_test_split(X, y,random_state = 0)<jupyter_output><empty_output><jupyter_text>We build a RandomForest similarly to how we built a decision tree in scikit-learn.<jupyter_code>from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_absolute_error

forest_model = RandomForestRegressor()
forest_model.fit(train_X, train_y)
melb_preds = forest_model.predict(val_X)
print(mean_absolute_error(val_y, melb_preds))<jupyter_output>205947.41056595655<jupyter_text>Conclusion There is likely room for further improvement, but this is a big improvement over the best decision tree error of 250,000. There are parameters which allow you to change the performance of the Random Forest much as we changed the maximum depth of the single decision tree. But one of the best features of Random Forest models is that they generally work reasonably even without this tuning.You'll soon learn the XGBoost model, which provides better performance when tuned well with the right parameters (but which requires some skill to get the right model parameters). Your TurnRun the RandomForestRegressor on your data. You should see a big improvement over your best Decision Tree models.  Continue You will see more big improvements in your models as soon as you start the  Intermediate track in*Learn Machine Learning* . But you now have a model that's a good starting point to compete in a machine learning competition!Follow **[these steps](https://www.kaggle.com/dansbecker/submitting-from-a-kernel/)** to make submissions for your current model. Then you can watch your progress in subsequent steps as you climb up the leaderboard with your continually improving models.<jupyter_code>import pandas as pd
from sklearn.model_selection import train_test_split

data = pd.read_csv('./data/house-prices-advanced-regression-techniques/train.csv')

X = data[["LotArea", "YearBuilt", "1stFlrSF", "2ndFlrSF", "FullBath", "BedroomAbvGr", "TotRmsAbvGrd"]]
y = data['SalePrice']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_absolute_error

rmf_reg = RandomForestRegressor(random_state=42)
rmf_reg.fit(X_train, y_train)
y_pred = rmf_reg.predict(X_test)
mean_absolute_error(y_test, y_pred)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.05998229305002213, 'fraction_numerical': 0.0073041168658698535, 'mean_word_length': 5.587463556851312, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '216470', 'license': 'MIT', 'n_tokens_mistral': 1272, 'n_tokens_neox': 1211, 'n_words': 542, 'path': 'tutorials/kaggle/machine-learning/random-forests.ipynb', 'repo_name': 'ivan-magda/ml-notebooks', 'size': '4518'}