<jupyter_start><jupyter_text>TPOT DemoSmall demo notebook to create a trained model using TPOT. SetupFirst we need to import our dependencies and load the Boston housing price dataset.<jupyter_code>from pathlib import Path
import json
import pandas as pd
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split
from tpot import TPOTRegressor

boston = load_boston()<jupyter_output><empty_output><jupyter_text>DatasetWe're using a common dataset comparing housing prices in Boston in the 1970s. Scikit-Learn provides a description of the dataset and its training features:<jupyter_code>print(boston.DESCR)<jupyter_output>.. _boston_dataset:

Boston house prices dataset
---------------------------

**Data Set Characteristics:**  

    :Number of Instances: 506 

    :Number of Attributes: 13 numeric/categorical predictive. Median Value (attribute 14) is usually the target.

    :Attribute Information (in order):
        - CRIM     per capita crime rate by town
        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.
        - INDUS    proportion of non-retail business acres per town
        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
        - NOX      nitric oxides concentration (parts per 10 million)
        - RM       average number of rooms per dwelling
        - AGE      proportion of owner-occupied units built prior to 1940
        - DIS      weighted distances to five Boston employment centres
        - RAD      index of accessibility to radial highways
        - TAX      full-value property-tax rate per $10,000
        - PTRATIO  pu[...]<jupyter_text>Let's look at the dataset a bit before building our model by loading it into a DataFrame. We need to add the home values back into the data rows.<jupyter_code>bdf = pd.DataFrame(data=boston.data, columns=boston.feature_names)
bdf['MDEV'] = pd.Series(boston.target)

bdf.head(10)<jupyter_output><empty_output><jupyter_text>TrainingWe need to randomly divide our data into training and testing sets. Testing a model on training data causes overfitting and reduces the model's usefulness beyond data found in the dataset.<jupyter_code>TRAIN_SIZE = 0.8

X_train, X_test, Y_train, Y_test = train_test_split(
    boston.data,
    boston.target,
    train_size=TRAIN_SIZE,
    test_size=1-TRAIN_SIZE,
)<jupyter_output><empty_output><jupyter_text>Now we train the model. TPOT will run the data through many possible model types and metadata configurations. This should give us a finished model that fits the data the best without having to select that information ourselves.Because the predicted value is a dollar amount (a non-discreate value), we use a regression model instead of a classification model.<jupyter_code>model = TPOTRegressor(generations=10, population_size=50, verbosity=2, n_jobs=1)
model.fit(X_train, Y_train)<jupyter_output><empty_output><jupyter_text>Score and ExportNow that our model is trained, let's see the final score. Because this is a regression model, the closer the score is to zero, the more accurate our model is.<jupyter_code>print(model.score(X_test, Y_test))<jupyter_output>-10.758821081365664<jupyter_text>Now that TPOT has found a good model to use, we need to export the code to run that model. TPOT makes it easy to save this code in a Python file. It even plays well with `Path` objects 😃<jupyter_code>output_code = Path('pipeline.py')
model.export(output_code)<jupyter_output><empty_output><jupyter_text>That's great for ML pipelines, but we want to see that code. Let's just read it back in.<jupyter_code>print(output_code.open().read())<jupyter_output>import numpy as np
import pandas as pd
from sklearn.ensemble import AdaBoostRegressor, RandomForestRegressor
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline, make_union
from tpot.builtins import StackingEstimator

# NOTE: Make sure that the class is labeled 'target' in the data file
tpot_data = pd.read_csv('PATH/TO/DATA/FILE', sep='COLUMN_SEPARATOR', dtype=np.float64)
features = tpot_data.drop('target', axis=1).values
training_features, testing_features, training_target, testing_target = \
            train_test_split(features, tpot_data['target'].values, random_state=None)

# Average CV score on the training set was:-10.222908257128386
exported_pipeline = make_pipeline(
    StackingEstimator(estimator=AdaBoostRegressor(learning_rate=0.1, loss="square", n_estimators=100)),
    StackingEstimator(estimator=RandomForestRegressor(bootstrap=False, max_features=0.15000000000000002, min_samples_leaf=8, min_samples_split=9, n_estimators=100)),
  [...]
{'fraction_non_alphanumeric': 0.07537473233404711, 'fraction_numerical': 0.022483940042826552, 'mean_word_length': 4.675577156743621, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 30, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '645973', 'license': 'MIT', 'n_tokens_mistral': 1475, 'n_tokens_neox': 1348, 'n_words': 483, 'path': 'tpot-intro/Boston Model.ipynb', 'repo_name': 'rserran/codex-python-workshop', 'size': '4670'}