<commit_before># _Design:_ xgboost on sqlflow

## Overview

This is a design doc about why and how to support running xgboost via sqlflow as a machine learning estimator.

We propose to build a lightweight python template for xgboost on basis of `xgblauncher`,
an incubating xgboost wrapper in [ant-xgboost](https://github.com/alipay/ant-xgboost).

## Context

Gradient boosting machine (GBM) is a widely used (supervised) machine learning method, 
which trains a bunch of weak learners, typically decision trees, 
in a gradual, additive and sequential manner. 
A lot of winning solutions of data mining and machine learning challenges, 
such as : Kaggle, KDD cup, are based on GBM or related techniques.

There exists a lot of GBM frameworks (implementations), we propose to use [xgboost](https://xgboost.ai/) as backend of sqlflow, 
who is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable, 
often regarded as one of the best GBM framework.


## _Proposed Solution:_ ant-xgboost on sqlflow
   
We propose to use [ant-xgboost](https://github.com/alipay/ant-xgboost) as backend,
which is consistent with [xgboost](https://github.com/dmlc/xgboost) in kernel level. 
Because in `ant-xgboost`, there exists an incubating module named [xgblauncher](https://github.com/alipay/ant-xgboost/tree/ant_master/xgboost-launcher), 
an extendable, cloud-native xgboost based machine learning pipeline. 
Comparing to python API provided by `xgboost`, it is easier to build a python code template for xgboost task launching on basis of `xgblauncher`.

### User Experience
    
In terms of sqlflow users, xgboost is an alternative `Estimator` like `TensorFlow Estimators`. 
Working with xgboost is quite similar to working with TensorFlow Estimators; just change `TRAIN DNNClassifier` into `TRAIN XGBoostEstimator`. 

In addition, xgboost specific parameters can be configured in the same way as TensorFlow parameters. 

Below is a demo about training/predicting via xgboost :

```sql
// sample clause of train
select 
    c1, c2, c3, c4, c5 as class
from kaggle_credit_fraud_training_data<commit_msg> Fix a few typos (#652)<commit_after># _Design:_ xgboost on sqlflow

## Overview

This is a design doc about why and how to support running xgboost via sqlflow as a machine learning estimator.

We propose to build a lightweight python template for xgboost on basis of `xgblauncher`,
an incubating xgboost wrapper in [ant-xgboost](https://github.com/alipay/ant-xgboost).

## Context

Gradient boosting machine (GBM) is a widely used (supervised) machine learning method, 
which trains a bunch of weak learners, typically decision trees, 
in a gradual, additive and sequential manner. 
A lot of winning solutions of data mining and machine learning challenges, 
such as : Kaggle, KDD cup, are based on GBM or related techniques.

There exists a lot of GBM frameworks (implementations), we propose to use [xgboost](https://xgboost.ai/) as backend of sqlflow, 
which is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable, 
often regarded as one of the best GBM frameworks.


## _Proposed Solution:_ ant-xgboost on sqlflow
   
We propose to use [ant-xgboost](https://github.com/alipay/ant-xgboost) as backend,
which is consistent with [xgboost](https://github.com/dmlc/xgboost) in kernel level. 
Because in `ant-xgboost`, there exists an incubating module named [xgblauncher](https://github.com/alipay/ant-xgboost/tree/ant_master/xgboost-launcher), 
an extendable, cloud-native xgboost based machine learning pipeline. 
Comparing to python API provided by `xgboost`, it is easier to build a python code template for xgboost task launching on basis of `xgblauncher`.

### User Experience
    
In terms of sqlflow users, xgboost is an alternative `Estimator` like `TensorFlow Estimators`. 
Working with xgboost is quite similar to working with TensorFlow Estimators; just change `TRAIN DNNClassifier` into `TRAIN XGBoostEstimator`. 

In addition, xgboost specific parameters can be configured in the same way as TensorFlow parameters. 

Below is a demo about training/predicting via xgboost :

```sql
// sample clause of train
select 
    c1, c2, c3, c4, c5 as class
from kaggle_credit_fraud_training_data
{'fraction_non_alphanumeric': 0.06638714185883997, 'fraction_numerical': 0.003028185418122525, 'mean_word_length': 4.771505376344086, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3593696', 'n_tokens_mistral': 1280, 'n_tokens_neox': 1200, 'n_words': 554}