<jupyter_start><jupyter_text>Comparing Machine Learning Predcitions Using SubplotsAfter performing different model predictions, pherhaps using different parameters or algorithms we are often left with a large set of  predictions that we want to compare. The standard technique is to plot the predicted values ontop of the expected  values in order to quickly see if there is a good fit and that our model has managed to capture the general structure. This last bit is why a visual insepction is very beneficial even if the model has returned a high value for the R-suqare evaluation metric.     Automating The Plotting ProcedureIf we have more than just 2 models that we are going to compare many times, then it can be very time consuming to repeatedly modify and add subplots. For this case, the best thing to do is to create a personal plotting function, or to import existing libraries to do the job for you.<jupyter_code># LIBRARIES. 
# Special Libraries. 
import os 

# Numerical and Data Libraries. 
import pandas as pd
import numpy as np 
import matplotlib.pyplot as plt

# Machine Learning Libraries
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn import linear_model
from sklearn.inspection import permutation_importance
from sklearn.ensemble import IsolationForest #Outlier Removal. 

# Machine Learning Models. 
from sklearn.neighbors import KNeighborsRegressor
from sklearn.neural_network import MLPRegressor
import xgboost as xgb 
from sklearn import svm 
from sklearn import tree
from sklearn.ensemble import RandomForestRegressor
from sklearn import linear_model
from sklearn.linear_model import ElasticNet

plt.style.use("seaborn-darkgrid")
# Load Example Data 
from sklearn.datasets import  load_diabetes
data = load_diabetes()

# Create DataFrame from  Data. 
df_data = pd.DataFrame(data=data["data"], columns=data["feature_names"])

# Get an Overview of the data. 
df_data<jupyter_output><empty_output><jupyter_text>Define Test and Training Data<jupyter_code># Chose Prediction Variable in your model.
predVar = "s1"
y = df_data[predVar]

# Variables that you do not want to include in your model. 
drop_col = [predVar]
x = df_data.drop(drop_col,axis = 1)
                                              
# Split data into random training and test-sets. 
x_train, x_test, y_train, y_test = train_test_split(x,y,  test_size=0.2)<jupyter_output><empty_output><jupyter_text>Define All Machine Learning Models You Want To Compare<jupyter_code># Store Models With Names as (name, model). 
machine_models = []

lm_model = linear_model.LinearRegression()
machine_models.append(("LM",lm_model ))

svm_model = svm.SVR()
machine_models.append(("SVM",svm_model ))

mlp_model = MLPRegressor(hidden_layer_sizes=400, max_iter = 5000)
machine_models.append(("MLP",mlp_model ))

kNeighbor_model = KNeighborsRegressor()
machine_models.append(("K-Neighbor",kNeighbor_model ))

xgb_model = xgb.XGBRegressor()
machine_models.append(("XGBoost",xgb_model ))

DTR_model = tree.DecisionTreeRegressor()
machine_models.append(("Decision Tree",DTR_model ))

elasticNet_model = ElasticNet()
machine_models.append(("elasticNet",elasticNet_model ))

RForest_model = RandomForestRegressor()
machine_models.append(("RForest_model",RForest_model ))<jupyter_output><empty_output><jupyter_text>Loop Over All Models, Fit & Predict.<jupyter_code># Loop over Models. Fit to training set. 
predictions = []
for model_unit in machine_models:
    name = model_unit[0]
    model = model_unit[1]

    # Fit and Predict the values from the training and test sets. 
    model.fit(x_train,y_train)
    prediction = model.predict(x_test)

    # Store Predictions in seperate list for later plotting. 
    predictions.append((name, prediction))
    continue 



def subPlotter(predList,y_blind, title, var, ylabel):
    from sklearn.metrics import r2_score
    """
    Parameters
    ----------
    predList : List of 2-tpuples (name, modelPredictions). 
        Example: [("XGBoost", y_xgb), ("MLP", y_ann), ("Rforest", y_forest)]

    y_blind : 1D List/Vector. 
        Actual Predictions 
    title : String
        Description of overall plot. 
    var : String
        x-axis label.

    Returns
    -------
    Plot.
        A plot of modelPredictions. 

    """
    # How many subplots do we have? 
    plotSize = len(predList)
    
    # Initilize Subplot figure and axis  
    fig, axes = plt.subplots(1,plotSize ,figsize = (20,10) )
    fig.suptitle(title)
    plt.ylabel(ylabel)
    
    # Loop Over All Model Predictions and add subplot.  
    for (i, prediction) in enumerate(predList):
        
        # Touple --> (name, y_pred)
        modelName = prediction[0]
        y_pred = prediction[1]
        
        # Add Subplot i. 
        axes[i].plot( y_blind, y_blind.index, ".")
        axes[i].plot( y_pred, y_blind.index, ".")
        axes[i].set_title(modelName)
        axes[i].set_xlabel(var)
       
        # Add Textbox with R2 Value At Top of each subplot. 
        R2 = round(r2_score(y_blind, y_pred), 2)
        text_box = f'R2 = {R2}'
        props = dict(boxstyle='round', facecolor='wheat', alpha=0.5)
        axes[i].text(0.35, 0.99, text_box, transform=axes[i].transAxes, fontsize=14,
                verticalalignment='top', bbox=props)
        
        # Add names to seperate predicted data from known data. 
        names = ["Actual", "Predicted"]
        axes[i].legend(names, loc = "lower right")
        continue 
    plt.show()
    return print("Subplot have been performed for model predictions.")<jupyter_output><empty_output><jupyter_text>Compare And Plot All Models In The Same Plot<jupyter_code># Plot All the Predictions Above we got Above.  
predList = predictions
title = "Model Comparisons"
var = "s1"
ylabel = "Y-AXIS "

# Normally the blind is different than the test. 
y_blind = y_test
subPlotter(predList,y_blind, title, var,ylabel)
 
# Exstract Mean Feature Importance For Each Variable. 
feat_importance = permutation_importance(lm_model,x_test, y_test)
mean_importance = feat_importance["importances_mean"]

# Use Pandas Series Inbuilt Method to plot Sorted BarChart. 
feat_importances = pd.Series(mean_importance, index=x_test.columns)
feat_importances.nlargest(20).plot(kind='barh')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08703908846336446, 'fraction_numerical': 0.006804874188953949, 'mean_word_length': 3.9336455893832944, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 22, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '620982', 'license': 'Apache-2.0', 'n_tokens_mistral': 2069, 'n_tokens_neox': 1947, 'n_words': 601, 'path': 'MachineLearning_ComparingModels.ipynb', 'repo_name': 'efl008/DataScience', 'size': '6319'}