<jupyter_start><jupyter_text>Getting Started Tutorial with TMLT (Tabular ML Toolkit)> A tutorial on getting started with TMLT (Tabular ML Toolkit)> tabular_ml_toolkit is a helper library to jumpstart your machine learning project based on Tabular or Structured data.> It comes with model parallelism and cutting edge hyperparameter search techniques.> Under the hood TMLT uses optuna, xgboost and scikit-learn pipelines  Install `pip install -U tabular_ml_toolkit`  How to Best Use tabular_ml_toolkit Start with your favorite model and then just simply create tmlt with one API *For example, Here we are using XGBRegressor on  [Melbourne Home Sale price data](https://www.kaggle.com/estrotococo/home-data-for-ml-course)*<jupyter_code>from tabular_ml_toolkit.tmlt import *
from sklearn.metrics import mean_absolute_error
import numpy as np
from xgboost import XGBRegressor
# Dataset file names and Paths
DIRECTORY_PATH = "input/home_data/"
TRAIN_FILE = "train.csv"
TEST_FILE = "test.csv"
SAMPLE_SUB_FILE = "sample_submission.csv"
OUTPUT_PATH = "output/"<jupyter_output><empty_output><jupyter_text>Just point tmlt in the direction of your data  let it know what are idx and target columns in your tabular data  and what kind of problem type you are trying to resolve<jupyter_code>%%time
# tmlt
tmlt = TMLT().prepare_data(
    train_file_path= DIRECTORY_PATH+TRAIN_FILE,
    test_file_path= DIRECTORY_PATH+TEST_FILE,
    idx_col="Id", target="SalePrice",
    random_state=42,
    problem_type="regression")

# TMLT currently only supports below problem_type:

# "binary_classification"
# "multi_label_classification"
# "multi_class_classification"
# "regression"
print(type(tmlt.dfl.X))
print(tmlt.dfl.X.shape)
print(type(tmlt.dfl.y))
print(tmlt.dfl.y.shape)
print(type(tmlt.dfl.X_test))
print(tmlt.dfl.X_test.shape)
tmlt.dfl.X<jupyter_output><empty_output><jupyter_text>Training  Now PreProcess X_train, X_validNOTE: Preprocessing gives back numpy arrays for pandas dataframe<jupyter_code>%%time
X_np, y_np, X_test_np = tmlt.pp_fit_transform(tmlt.dfl.X, tmlt.dfl.y, tmlt.dfl.X_test)
print(X_np.shape)
print(type(X_np))
print(y_np.shape)
print(type(y_np))
print(X_test_np.shape)
print(type(X_test_np))<jupyter_output><empty_output><jupyter_text>create train valid dataframes for quick preprocessing and training<jupyter_code>%%time
# create train, valid split to evaulate model on valid dataset
X_train_np, X_valid_np,  y_train_np, y_valid_np =  tmlt.dfl.create_train_valid(X_np, y_np, valid_size=0.2)
print(X_train_np.shape)
print(X_valid_np.shape)
print(y_train_np.shape)
print(y_valid_np.shape)<jupyter_output><empty_output><jupyter_text>Training  Create a base xgb classifier model with your best guess params<jupyter_code>xgb_params = {
    'learning_rate':0.1,
    'use_label_encoder':False,
    'eval_metric':'rmse',
    'random_state':42,
    # for GPU
#     'tree_method': 'gpu_hist',
#     'predictor': 'gpu_predictor',
}
# create xgb ml model
xgb_model = XGBRegressor(**xgb_params)
%%time
# Now do model training
xgb_model.fit(X_train_np, y_train_np,
              verbose=False,
              #detect & avoid overfitting
              eval_set=[(X_train_np, y_train_np), (X_valid_np, y_valid_np)],
              eval_metric="mae",
              early_stopping_rounds=300
             )

#predict
preds = xgb_model.predict(X_valid_np)
print('X_valid MAE:', mean_absolute_error(y_valid_np, preds))<jupyter_output><empty_output><jupyter_text>In background `prepare_data` method loads your input data into Pandas DataFrame, seprates X(features) and y(target), preprocess all numerical and categorical type data found in these DataFrames using scikit-learn pipelines. Then it bundle preprocessor and data return a TMLT object, this class instance has dataframeloader, preprocessor instances.The `create_train_valid` method use valid_size to split X(features) into X_train, y_train, X_valid and y_valid DataFrames, so you can call fit methods on X_train and y_train and predict methods on X_valid or X_test.Please check detail documentation and source code for more details. *NOTE: If you want to customize data and preprocessing steps you can do so by using `DataFrameLoader` and `PreProessor` classes. Check detail documentations for these classes for more options.*  To see more clear picture of model performance, Let's do a quick Cross Validation on our Pipeline  Make sure to PreProcess the data<jupyter_code>%%time
# Now do cross_validation
scores = tmlt.do_cross_validation(X_np, y_np, xgb_model, scoring='neg_mean_absolute_error', cv=5)

print("scores:", scores)
print("Average MAE score:", scores.mean())<jupyter_output><empty_output><jupyter_text>*MAE did came out slightly bad with cross validation* *Let's see if we can improve our cross validation score with hyperparams tunning* **We are using optuna based hyperparameter search here!****TMLT has inbuilt xgb optuna optimization helper method!**<jupyter_code># **Just make sure to supply an output directory path so hyperparameter search is saved**
study = tmlt.do_xgb_optuna_optimization(optuna_db_path=OUTPUT_PATH, opt_timeout=10)
print(study.best_trial)<jupyter_output><empty_output><jupyter_text>Let's use our newly found best params to update the model on sklearn pipeline<jupyter_code>xgb_params.update(study.best_trial.params)
print("xgb_params", xgb_params)
updated_xgb_model = XGBRegressor(**xgb_params)<jupyter_output><empty_output><jupyter_text>Now, Let's use 5 K-Fold Training on this Updated XGB model with best params found from Optuna search<jupyter_code>%%time

# k-fold training
xgb_model_metrics_score, xgb_model_test_preds = tmlt.do_kfold_training(X_np, y_np, model=updated_xgb_model, 
X_test=X_test_np, n_splits=5, kfold_metric=mean_absolute_error, eval_metric = 'mae')
# predict on test dataset
if xgb_model_test_preds is not None:
    print(xgb_model_test_preds.shape)<jupyter_output><empty_output><jupyter_text>You can even improve metrics score further by running Optuna search for longer time or rerunning the study, check documentation for more details<jupyter_code>#fin<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08368887430259271, 'fraction_numerical': 0.002625533311453889, 'mean_word_length': 6.281959378733572, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 45, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '243281', 'license': 'Apache-2.0', 'n_tokens_mistral': 2070, 'n_tokens_neox': 2012, 'n_words': 570, 'path': 'index.ipynb', 'repo_name': 'psmathur/tabular_ml_toolkit', 'size': '6094'}