<jupyter_start><jupyter_text>Train a model to predict a color type<jupyter_code># eliminate library warnings
import warnings
warnings.simplefilter('ignore')

import pandas as pd
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import matplotlib.lines as mlines

# set matplotlib outputs/plots to be embeded inline to notebook cells
%matplotlib inline
# read data
df = pd.read_csv('data/warm_cold_colors.csv')
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 136 entries, 0 to 135
Data columns (total 4 columns):
r          136 non-null int64
g          136 non-null int64
b          136 non-null int64
is_warm    136 non-null int64
dtypes: int64(4)
memory usage: 4.3 KB<jupyter_text>It is very usefull to downcast the data types from the default ones to reduce the memory consumption and consequentially to speed-up computation process<jupyter_code># cast the data types
df[['r', 'g', 'b']] = df[['r', 'g', 'b']].astype(np.int16)
df['is_warm'] = df['is_warm'].astype(np.int8)

df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 136 entries, 0 to 135
Data columns (total 4 columns):
r          136 non-null int16
g          136 non-null int16
b          136 non-null int16
is_warm    136 non-null int8
dtypes: int16(3), int8(1)
memory usage: 1.0 KB<jupyter_text>In our case, we reduced the memory consumtion by the factor of **4**(!) even though it wasn't critical since we have a tiny data set, however in case of 1M+ records, it is a reasonable first step prior to building a machine learning model. In fact, types downcasting is used for tensorflow lite as a part of the post-training quantization (see details here).<jupyter_code># check the classes distribution skewness
df[['is_warm', 'r']].groupby(['is_warm'])\
                    .count()\
                    .reset_index()<jupyter_output><empty_output><jupyter_text>Explorative Data Analysis - EDA<jupyter_code>axes_combinations = [('r', 'g'), 
                     ('r', 'b'), 
                     ('g', 'b')]

color_scale = {'Cool': 'blue', 
               'Warm':'red'}
fig, axes = plt.subplots(nrows=1, ncols=3, figsize=(20, 10))

for i, ax in enumerate(axes.flat):
    x, y = axes_combinations[i]
    
    im = ax.scatter(x=df[x], 
                    y=df[y], 
                    c=df['is_warm'], 
                    cmap=matplotlib.colors.ListedColormap(color_scale.values()))

    ax.set_xlabel(x)
    ax.set_ylabel(y)
    ax.add_line(mlines.Line2D([0,255], [0,255], 
                              color='k'))
    
cb = fig.colorbar(im, 
                  ax=axes.ravel().tolist(), 
                  orientation='horizontal', 
                  shrink=.2) 
cb.ax.set_title('Color Type', pad=10)
cb.set_ticks([.25,.75])
cb.set_ticklabels(list(color_scale.keys()))

plt.show()<jupyter_output><empty_output><jupyter_text>Modelling  MetricsHow close to reality a model's prediction can be; for a classificaiton problem, consufion matrix is usually used.<jupyter_code>%%html
<img src="https://upload.wikimedia.org/wikipedia/commons/2/26/Precisionrecall.svg" alt="Precisionrecall.svg" height="480" width="264">
<div style="text-align:center">
By <a href="//commons.wikimedia.org/wiki/User:Walber" title="User:Walber">Walber</a> - <span class="int-own-work" lang="en">Own work</span>, 
<a href="https://creativecommons.org/licenses/by-sa/4.0" title="Creative Commons Attribution-Share Alike 4.0">CC BY-SA 4.0</a>, <a href="https://commons.wikimedia.org/w/index.php?curid=36926283">Link</a>
</div><jupyter_output><empty_output><jupyter_text>In order to reduce the flase negative and false positive rate, F1 score to be used as the model validation metric:             $F_{1} = \frac{2}{\frac{1}{Recall} + \frac{1}{Precision}}$                 $Recall = \frac{TP}{TP+FN}$     describes what fraction of all true "positive" points did a model predict correctly             $Precision = \frac{TP}{TP+FP}$     descibes what fraction of all predicted "Positive" points did a model predict correctly Another widely used abused metric, accuracy:    $Accuracy = \frac{TP+TN}{TP+FP+TN+FN}$<jupyter_code>def eval_metrics(y_true: np.array, 
                 y_pred: np.array) -> dict:
    """
    Function to calculate classification metrics
    
        Args:
            y_true: real labels
            y_pred: predicitons
    
        Returs:

            dict with the keys:
                confusion_matrix - confusion matrix
                accuracy = (TP+TN)/(TP+TN+FP+FN)
                precision = TP/(TP+FP)
                recall = TP/(TP+FN)
                f1_score = 2/(1/recall+1/precision)    
    """
    
    assert len(y_true) != 0, "Empty array, please check input"
    
    true_positives = np.where(y_true==1)[0]    
    TP = sum(y_pred[true_positives]==1)
    FN = sum(y_pred[true_positives]==0)
    
    true_negatives = np.where(y_true==0)[0]
    FP = sum(y_pred[true_negatives]==1)
    TN = sum(y_pred[true_negatives]==0)
    
    accuracy = (TP + TN) / len(y_true)
    
    recall = precision = f1_score = None
    
    if FN > 0 or TP > 0:
        recall = float(TP / (TP + FN))
        
    if FP > 0 or TP > 0:
        precision = float(TP / (TP + FP))

    if recall and precision:
        f1_score = 2/(1/recall+1/precision)    
        
    return {
        "confusion_matrix": 
            {
                "TP": TP, "FP": FP, 
                "FN": FN, "TN": TN
            },
            "accuracy": accuracy,
            "recall": recall,
            "precision": precision,
            "f1_score": f1_score
           }<jupyter_output><empty_output><jupyter_text>Baseline  As a baseline, we can set the condition: ```javascriptif r > g > b:    color_type = 'Warm'else:    color_type = 'Cool'```<jupyter_code>class model_baseline:
    """ Baseline model """
    
    import pandas as pd
    
    def __init__(self):
        pass
    
    @classmethod
    def _rule(cls, row: pd.DataFrame) -> int:
        """ Model rule
        
            Args:
                Row: pd.DataFrame row

            Return:
                int
        """

        if row['r'] > row['g'] > row['b']:
            return 1

        return 0

    
    def predict(cls, X: pd.DataFrame) -> pd.core.series.Series:
        """ Function to run a base line prediction

            Args:
                X: input data

            Returns:
                array
        """
    
        return X.apply(lambda row: cls._rule(row), axis=1)
# baseline model
model_v1 = model_baseline()
y_predict_baseline = model_v1.predict(df[['r', 'g', 'b']])

# baseline model evaluation
eval_metrics(df['is_warm'], y_predict_baseline)<jupyter_output><empty_output><jupyter_text>So without any machine learning, we were managed to get the F1 score of 0.74 :) Let's assess out "model" using a random data point:<jupyter_code># test point -> a color of the "Cool" type/class 0

test_point = pd.DataFrame({'r': [8], 'g': [103], 'b': [203]})
model_v1.predict(test_point).squeeze()<jupyter_output><empty_output><jupyter_text>Let's deploy the baseline model as a POC  We can improve our ML service by improving our model's accuracy  Data preparesion<jupyter_code>from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
seed = 2019
df = df.sample(frac=1, random_state=seed)\
       .reset_index(drop=True)
scale = MinMaxScaler()
X = df.drop('is_warm', axis=1)
X_scaled = scale.fit_transform(X)
x_train, x_test, y_train, y_test = train_test_split(X_scaled, df['is_warm'], 
                                                    test_size=0.2, 
                                                    random_state=seed)<jupyter_output><empty_output><jupyter_text>XGBoost<jupyter_code>from xgboost import XGBClassifier as xgb_class 
x_train, x_test, y_train, y_test = train_test_split(df.drop('is_warm', axis=1), df['is_warm'], 
                                                    test_size=0.2, 
                                                    random_state=seed)
params = {
    "objective": 'binary:logistic',
    "learning_rate": 0.5, 
    "n_estimators": 100, 
    "max_depth": 3,
    "n_jobs": 4,
    "silent": False, 
    "subsample": 0.8,
    "random_state": seed
}

model_v2 = xgb_class(**params)

model_v2.fit(x_train, y_train, verbose=True)
y_predict_xgb = model_v2.predict(df.drop('is_warm', axis=1))
eval_metrics(df['is_warm'], y_predict_xgb)
# point test

model_v2.predict(test_point).squeeze()<jupyter_output><empty_output><jupyter_text>0.97 is quite an improvement, so let's dump the model and re-deploy the service<jupyter_code>import pickle

def save_object(obj, filename):
    """
    Function to save/pickle python object

        Args:
            filename: str path to pickle file
    """

    with open(filename, 'wb') as output:
        pickle.dump(obj, output, -1)
save_object(model_v2, '../model/v2/model_v2.sav')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12393449977568416, 'fraction_numerical': 0.021982951996410947, 'mean_word_length': 2.701535907015359, 'pattern_counts': {'":': 17, '<': 56, '<?xml version=': 0, '>': 68, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '552656', 'license': 'MIT', 'n_tokens_mistral': 3138, 'n_tokens_neox': 2893, 'n_words': 739, 'path': 'server/color_type/ml/train/ml_steps.ipynb', 'repo_name': 'kislerdm/color_theory_app', 'size': '8916'}