<jupyter_start><jupyter_text>Table of Contents* **Getting Started**: importing/setting up the data* **The Long Way** : manually doing all the typical steps of pre-processing/training/predicting, using **oo-learning** classes    * Splitting    * Transforming    * Training    * Predicting    * Evaluating* **The Short Way**: presents the **`ModelTrainer`** class, which encapsulates all of the previously mentioned steps. Note: this notebook is meant to be a demo of some of the capabilities of **`oo-learning`** (https://github.com/shane-kercheval/oo-learning); it is not meant to show the best approach to exploring/cleaning/modeling this particular dataset. Also, with most graphs (e.g. correlations/box-plots/etc.) I will spend very little time commenting on the significance of any interesting or patterns. Again, the intent is to show a demo, not a guide to data analysis.   Getting StartedIn the last Notebook, https://github.com/shane-kercheval/oo-learning/blob/master/examples/classification-titanic/1-Exploring%20the%20Titanic%20Dataset.ipynb, I showed how to get started and explore the dataset. In this Notebook, we'll look at basic operations like training models, transformations, etc.  Set Up Environment<jupyter_code># !pip install oolearning --upgrade
from oolearning import *
import pandas as pd
import matplotlib.pyplot as plt

%matplotlib inline
pd.set_option('display.max_colwidth', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)
width = 10
plt.rcParams['figure.figsize'] = [width, width/1.333]<jupyter_output><empty_output><jupyter_text>Import Data `ExploreRegressionDataset` is a convenience class described in the [first notebook of this series](https://github.com/shane-kercheval/oo-learning/blob/master/examples/regression-insurance/1-Exploring.ipynb).<jupyter_code>csv_file = '../data/insurance.csv'
target_variable = 'expenses'

explore = ExploreRegressionDataset.from_csv(csv_file_path=csv_file,
                                                target_variable=target_variable)
explore.dataset.head()<jupyter_output><empty_output><jupyter_text>The Long Way The following sections provide a demo of the basic individual classes that are used when training a model. In the **`"The Short Way"`** section, next, I demo some of the classes that encapsulate the logic and take care of the mundane and repetitive tasks.    Splitting the Data Let's start out by splitting our data into a training set and holdout set.To do this, we can use, for example, the **`RegressionStratifiedDataSplitter`**, which as the name suggests, splits the data into training/holdout sets while maintaining a similar distribution (think histogram) of the target variable. Technically, the Splitter will give us the indexes that should go into each set, and we will make the split.<jupyter_code>splitter = RegressionStratifiedDataSplitter(holdout_ratio=0.20)  # set aside 20% of the data for the holdout set
training_indexes, holdout_indexes = splitter.split(target_values=explore.dataset.expenses)
training_y = explore.dataset.iloc[training_indexes][target_variable]
training_x = explore.dataset.iloc[training_indexes].drop(columns=target_variable)

holdout_y = explore.dataset.iloc[holdout_indexes][target_variable]
holdout_x = explore.dataset.iloc[holdout_indexes].drop(columns=target_variable)
training_x.shape
holdout_x.shape<jupyter_output><empty_output><jupyter_text>Let's check to make sure the Splitter did the other part of its job, which was to split the data in a stratified way, keeping the proportions of the target variable the same.<jupyter_code>training_y.hist()
holdout_y.hist()<jupyter_output><empty_output><jupyter_text>Pretty close!  Transforming the DataFrequently, we need to transform the data before we can train it with various models.Additionally, we'll want to *fit* transformations on the training set (i.e. the transformation object will extract the necessary attributes from the training set (e.g. median values for each column for a transformer that imputes missing values)) and then apply those same transformations to the holdout set (i.e. use the medians calculated/fitted on the training set, rather than the holdout set).Let's create a list of transformation objects that we want to apply to the training and holdout sets.Please see the [classification](http://localhost:8888/notebooks/examples/classification-titanic/2-Basic%20Modeling.ipynb) example for a more elaborate example and additional information.<jupyter_code>transformations = [DummyEncodeTransformer(CategoricalEncoding.DUMMY)]  # create dummy variables for categoric features<jupyter_output><empty_output><jupyter_text>In order to help manage all of the transformations we want to do (especially when we have more than one transformation object), we'll use the **`TransformerPipeline`** class and fit/transform the training data, then transform (without fitting) the holdout data.<jupyter_code>pipeline = TransformerPipeline(transformations=transformations)
transformed_training_data = pipeline.fit_transform(training_x)  # fit & transform
transformed_holdout_data = pipeline.transform(holdout_x)  # fit only
training_x.head()
transformed_training_data.head()<jupyter_output><empty_output><jupyter_text>Training a Model, & PredictingFinally, let's train a ridge regression model.The pattern for training a model includes two classes in most cases. One class specifies the model, in this case **`RidgeRegressor`**, and the other class species the hyper-parameters we want to use in the model. The corresponding class for the Ridge model is **`RidgeRegressorHP`**.The corresponding **`HP`** classes define the specific parameters that can be tuned, specified in the **`__init__(...)`** function. Instantiating the class without providing any values will result in default values being used.In most cases, the model classes, such as `RidgeRegressor`, are wrappers around `scikit-learn` classes, and the `HP` classes' default values for the hyper-parameters are the same values that the underlying `scikit-learn` classes use.<jupyter_code>model = RidgeRegressor()
model.train(data_x=transformed_training_data, data_y=training_y, hyper_params=RidgeRegressorHP())  # default hyper-parameters
predictions = model.predict(data_x=transformed_holdout_data)
predictions[0:10]<jupyter_output><empty_output><jupyter_text>Evaluating PerformanceWe've trained our model on the training data, and predicted on the holdout. Now we need some way of evaluating performance.We could use, for example, `scikit-learn`'s MSE or MAE score via `mean_squared_error` or `mean_absolute_error`.<jupyter_code>from sklearn.metrics import mean_squared_error, mean_absolute_error
import math

math.sqrt(mean_squared_error(y_true=holdout_y, y_pred=predictions))
mean_absolute_error(y_true=holdout_y, y_pred=predictions)<jupyter_output><empty_output><jupyter_text>**`Evaluator`** object Great.But we can also use, for example, `oo-learning`'s **`RegressionEvaluator`** class. This gives us common quality metrics as well as convenient graphs.<jupyter_code>evaluator = RegressionEvaluator()
evaluator.evaluate(actual_values=holdout_y, predicted_values=predictions)  # note, we gave it the raw DataFrame, not the converted classes
evaluator.all_quality_metrics
evaluator.plot_residuals_vs_fits()
evaluator.plot_predictions_vs_actuals()
evaluator.plot_residuals_vs_actuals()<jupyter_output><empty_output><jupyter_text>The Short WayWhat we did is pretty standard, and we'll probably do the same set of generic steps time and time again.**`ModelTrainer`** is a class the encapsulate the boring details of what we did, so we can do it a lot quicker and with more concise and descriptive code.The following code has the identical result as all the work we did above.<jupyter_code>trainer = ModelTrainer(model=RidgeRegressor(),
                       model_transformations=[DummyEncodeTransformer(CategoricalEncoding.DUMMY)],
                       splitter= RegressionStratifiedDataSplitter(holdout_ratio=0.2),
                       evaluator=RegressionEvaluator())
trainer.train_predict_eval(data=explore.dataset,
                           target_variable='expenses',
                           hyper_params=RidgeRegressorHP())

# access the holdout metrics (we also have `training_evaluator` available to us as well!) and get a summary
trainer.holdout_evaluator.all_quality_metrics
trainer.holdout_evaluator.plot_residuals_vs_fits()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07820437093916126, 'fraction_numerical': 0.003662138216184288, 'mean_word_length': 5.877335499593826, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '515643', 'license': 'MIT', 'n_tokens_mistral': 2419, 'n_tokens_neox': 2326, 'n_words': 818, 'path': 'examples/regression-insurance/2-Basic Modeling.ipynb', 'repo_name': 'shane-kercheval/oo-learning', 'size': '8465'}