<jupyter_start><jupyter_text>Dask and XGBoost<jupyter_code>%matplotlib inline<jupyter_output><empty_output><jupyter_text>The `dask-xgboost` library provides a small wrapper around `dask-xgboost` for passing dask objects to `xgboost`.As usual, we can create or attach to our cluster by creating a `Client`<jupyter_code>import os
from dask import compute, persist
from dask.distributed import Client, progress

client = Client(os.environ.get("DISTRIBUTED_ADDRESS"))  # connect to cluster<jupyter_output><empty_output><jupyter_text>We'll work through an example using the airlines dataset.For a recorded screen cast, see [here](https://www.youtube.com/watch?v=Cc4E-PdDSro).Our task is to pick whether or not a flight was delayed for more than 15 minutes.First, let's load the data.<jupyter_code>import dask.dataframe as dd

# Subset of the columns to use
cols = ['Year', 'Month', 'DayOfWeek', 'Distance',
        'DepDelay', 'CRSDepTime', 'UniqueCarrier', 'Origin', 'Dest']

# Create the dataframe
df = dd.read_csv('s3://dask-data/airline-data/2006.csv', usecols=cols,
                 storage_options={'anon': True})

frac = 0.01
df = df.sample(frac=frac) # we blow out ram otherwise

is_delayed = (df.DepDelay.fillna(16) > 15)

df['CRSDepTime'] = df['CRSDepTime'].clip(upper=2399)
del df['DepDelay']

df, is_delayed = persist(df, is_delayed)
progress(df, is_delayed)
df.head()
is_delayed.head()
df2 = dd.get_dummies(df.categorize()).persist()<jupyter_output><empty_output><jupyter_text>Split and TrainWe'll split the original dataset into train and test sets.The model will be fit on `data_train` and evaluated on `data_test`.<jupyter_code>data_train, data_test = df2.random_split([0.9, 0.1], 
                                         random_state=1234)
labels_train, labels_test = is_delayed.random_split([0.9, 0.1], 
                                                    random_state=1234)<jupyter_output><empty_output><jupyter_text>Now we can fit the model. This is exactly like normal, except you'll import `dask_xgboost`.It will take care of handing the data off to the distributed `xgboost` processes, and from there everything proceeds as normal.<jupyter_code>%%time
import dask_xgboost as dxgb
import xgboost as xgb

params = {'objective': 'binary:logistic', 'nround': 1000, 
          'max_depth': 16, 'eta': 0.01, 'subsample': 0.5, 
          'min_child_weight': 1}

bst = dxgb.train(client, params, data_train, labels_train)
bst<jupyter_output>CPU times: user 962 ms, sys: 569 ms, total: 1.53 s
Wall time: 17.2 s<jupyter_text>The `bst` object is just a reguler `xgboost.Booster`, so all the familar methods are available.<jupyter_code>import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(12, 8))

ax = xgb.plot_importance(bst, ax=ax, height=0.8, max_num_features=20)
ax.grid("off", axis="y")<jupyter_output><empty_output><jupyter_text>Let's get the predictions. The is the same as usual, except we're using `dxgb`.<jupyter_code>predictions = dxgb.predict(client, bst, data_test)
predictions = client.persist(predictions)
predictions<jupyter_output><empty_output><jupyter_text>The `predictions` object is a `dask.Series`, so it can be used in place of a numpy array in most places.<jupyter_code>from sklearn.metrics import roc_curve, auc
fig, ax = plt.subplots(figsize=(8, 8))
fpr, tpr, _ = roc_curve(labels_test, predictions)
ax.plot(fpr, tpr, lw=3,
        label='ROC Curve (ares = {:.2f})'.format(auc(fpr, tpr)))
ax.plot([0, 1], [0, 1], 'k--', lw=2)

ax.set(
    xlim=(0, 1),
    ylim=(0, 1),
    title="ROC Curve",
    xlabel="False Positive Rate",
    ylabel="True Positive Rate",
)
ax.legend();<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12582236842105263, 'fraction_numerical': 0.022203947368421052, 'mean_word_length': 4.810509554140127, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 33, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '253407', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1338, 'n_tokens_neox': 1270, 'n_words': 345, 'path': 'docs/source/examples/xgboost.ipynb', 'repo_name': 'jrbourbeau/dask-ml', 'size': '3648'}