<jupyter_start><jupyter_text><NAME>'s Safe Driving Prediction<NAME>, one of Brazil’s largest auto and homeowner insurance companies, completely agrees. Inaccuracies in car insurance company’s claim predictions raise the cost of insurance for good drivers and reduce the price for bad ones.In the [Porto Seguro Safe Driver Prediction competition](https://www.kaggle.com/c/porto-seguro-safe-driver-prediction), the challenge is to build a model that predicts the probability that a driver will initiate an auto insurance claim in the next year. While Porto Seguro has used machine learning for the past 20 years, they’re looking to Kaggle’s machine learning community to explore new, more powerful methods. A more accurate prediction will allow them to further tailor their prices, and hopefully make auto insurance coverage more accessible to more drivers.Lucky for you, a machine learning model was built to solve the Porto Seguro problem by the data scientist on your team. The solution notebook has steps to load data, split the data into test and train sets, train, evaluate and save a LightGBM model that will be used for the future challenges. Hint: use shift + enter to run the code cells below. Once the cell turns from [*] to [], you can be sure the cell has run.   Import Needed PackagesImport the packages needed for this solution notebook. The most widely used packages for machine learning for [scikit-learn](https://scikit-learn.org/stable/), [pandas](https://pandas.pydata.org/docs/getting_started/index.htmlgetting-started), and [numpy](https://numpy.org/). These packages have various features, as well as a lot of clustering, regression and classification algorithms that make it a good choice for data mining and data analysis. In this notebook, we're using a training function from [lightgbm](https://lightgbm.readthedocs.io/en/latest/index.html).<jupyter_code>import os
import numpy as np
import pandas as pd
import lightgbm
from sklearn.model_selection import train_test_split
import joblib
from sklearn import metrics<jupyter_output><empty_output><jupyter_text>Load DataLoad the training dataset from the ./data/ directory. Df.shape() allows you to view the dimensions of the dataset you are passing in. If you want to view the first 5 rows of data, df.head() allows for this.<jupyter_code>DATA_DIR = "../data"
data_df = pd.read_csv(os.path.join(DATA_DIR, 'porto_seguro_safe_driver_prediction_input.csv'))
print(data_df.shape)
data_df.head()<jupyter_output>(595212, 59)<jupyter_text>Split Data into Train and Validatation SetsPartitioning data into training, validation, and holdout sets allows you to develop highly accurate models that are relevant to data that you collect in the future, not just the data the model was trained on. In machine learning, features are the measurable property of the object you’re trying to analyze. Typically, features are the columns of the data that you are training your model with minus the label. In machine learning, a label (categorical) or target (regression) is the output you get from your model after training it.<jupyter_code>features = data_df.drop(['target', 'id'], axis = 1)
labels = np.array(data_df['target'])
features_train, features_valid, labels_train, labels_valid = train_test_split(features, labels, test_size=0.2, random_state=0)

train_data = lightgbm.Dataset(features_train, label=labels_train)
valid_data = lightgbm.Dataset(features_valid, label=labels_valid, free_raw_data=False)<jupyter_output><empty_output><jupyter_text>Train ModelA machine learning model is an algorithm which learns features from the given data to produce labels which may be continuous or categorical ( regression and classification respectively ). In other words, it tries to relate the given data with its labels, just as the human brain does.In this cell, the data scientist used an algorithm called [LightGBM](https://lightgbm.readthedocs.io/en/latest/), which primarily used for unbalanced datasets. AUC will be explained in the next cell.<jupyter_code>parameters = {
    'learning_rate': 0.02,
    'boosting_type': 'gbdt',
    'objective': 'binary',
    'metric': 'auc',
    'sub_feature': 0.7,
    'num_leaves': 60,
    'min_data': 100,
    'min_hessian': 1,
    'verbose': 4
}
    
model = lightgbm.train(parameters,
                           train_data,
                           valid_sets=valid_data,
                           num_boost_round=500,
                           early_stopping_rounds=20)<jupyter_output>[1]	valid_0's auc: 0.595844
Training until validation scores don't improve for 20 rounds
[2]	valid_0's auc: 0.605252
[3]	valid_0's auc: 0.612784
[4]	valid_0's auc: 0.61756
[5]	valid_0's auc: 0.620129
[6]	valid_0's auc: 0.622447
[7]	valid_0's auc: 0.622163
[8]	valid_0's auc: 0.622112
[9]	valid_0's auc: 0.622581
[10]	valid_0's auc: 0.622278
[11]	valid_0's auc: 0.622433
[12]	valid_0's auc: 0.623423
[13]	valid_0's auc: 0.623618
[14]	valid_0's auc: 0.62414
[15]	valid_0's auc: 0.624421
[16]	valid_0's auc: 0.624512
[17]	valid_0's auc: 0.625151
[18]	valid_0's auc: 0.62529
[19]	valid_0's auc: 0.625437
[20]	valid_0's auc: 0.62563
[21]	valid_0's auc: 0.625963
[22]	valid_0's auc: 0.626147
[23]	valid_0's auc: 0.626383
[24]	valid_0's auc: 0.626618
[25]	valid_0's auc: 0.626586
[26]	valid_0's auc: 0.626839
[27]	valid_0's auc: 0.626972
[28]	valid_0's auc: 0.626935
[29]	valid_0's auc: 0.626946
[30]	valid_0's auc: 0.627204
[31]	valid_0's auc: 0.627252
[32]	valid_0's auc: 0.627302
[33]	valid_0's auc: 0.62[...]<jupyter_text>Evaluate ModelEvaluating performance is an essential task in machine learning. In this case, because this is a classification problem, the data scientist elected to use an AUC - ROC Curve. When we need to check or visualize the performance of the multi - class classification problem, we use AUC (Area Under The Curve) ROC (Receiver Operating Characteristics) curve. It is one of the most important evaluation metrics for checking any classification model’s performance.<img src="https://www.researchgate.net/profile/Oxana_Trifonova/publication/276079439/figure/fig2/AS:614187332034565@1523445079168/An-example-of-ROC-curves-with-good-AUC-09-and-satisfactory-AUC-065-parameters.png"     alt="Markdown Monster icon"     style="float: left; margin-right: 12px; width: 320px; height: 239px;" /><jupyter_code>predictions = model.predict(valid_data.data)
fpr, tpr, thresholds = metrics.roc_curve(valid_data.label, predictions)
model_metrics = {"auc": (metrics.auc(fpr, tpr))}
print(model_metrics)<jupyter_output>{'auc': 0.6377511613946426}<jupyter_text>Save Model In machine learning, we need to save the trained models in a file and restore them in order to reuse it to compare the model with other models, to test the model on a new data. The saving of data is called Serializaion, while restoring the data is called Deserialization.<jupyter_code>model_name = "lgbm_binary_model.pkl"
joblib.dump(value=model, filename=model_name)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08876759551852915, 'fraction_numerical': 0.05960930767020971, 'mean_word_length': 4.9512820512820515, 'pattern_counts': {'":': 1, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 7, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '882', 'license': 'MIT', 'n_tokens_mistral': 2479, 'n_tokens_neox': 2160, 'n_words': 815, 'path': 'step0/porto-seguro-safe-driver-prediction-LGBM.ipynb', 'repo_name': 'kawo123/azure-mlops', 'size': '6974'}