<jupyter_start><jupyter_text>Neptune + LightGBM  Install dependencies<jupyter_code>! pip install graphviz==0.16 lightgbm==3.2.1 neptune-client neptune-lightgbm<jupyter_output><empty_output><jupyter_text>Import libraries<jupyter_code>import lightgbm as lgb
import neptune.new as neptune
import numpy as np

from neptune.new.integrations.lightgbm import NeptuneCallback, create_booster_summary
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split<jupyter_output><empty_output><jupyter_text>Create run<jupyter_code>run = neptune.init(
    project="common/lightgbm-integration",
    api_token="<PASSWORD>",
    name="train-cls",
    tags=["lgbm-integration", "train", "cls", "notebook"],
)<jupyter_output><empty_output><jupyter_text>Link above is a link to the run. Click on it, and leave the run tab open - you will get back to it when you start model training.Few explanations:1. You need to pass project to the `project`, parameter to inform Neptune where to log metadata. Project is a string of this form: `my_workspace/my_project`.1. There are more parameters to customize Neptune behavior, check [neptune.init() docs](https://docs.neptune.ai/api-reference/neptuneinit) for more details.----**Note**Instead of logging data to the public project `"common/lightgbm-integration"` as an anonymous user `"neptuner"` you can log it to your own project.To do that:1. Follow the [installation and setup](https://docs.neptune.ai/getting-started/installation) that will show you how to use individual, private api_token.1. Create new [private project](https://docs.neptune.ai/administration/workspace-project-and-user-management/projects).1. Pass this project name here, instead of `"common/lightgbm-integration"`.At this point you will be ready to log LightGBM runs to your own project :)  Create neptune callback<jupyter_code>neptune_callback = NeptuneCallback(run=run)<jupyter_output><empty_output><jupyter_text>This callback will do metadata logging during training. You will pass it to the LightGBM `train()` function.It also works with `cv()` function and sklearn-like API of the LightGBM (in this case you pass it to the `fit()` function).  Prepare data and define parameters<jupyter_code># Prepare data
X, y = load_digits(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=123
)
lgb_train = lgb.Dataset(X_train, y_train)
lgb_eval = lgb.Dataset(X_test, y_test, reference=lgb_train)

# Define parameters
params = {
    "boosting_type": "gbdt",
    "objective": "multiclass",
    "num_class": 10,
    "metric": ["multi_logloss", "multi_error"],
    "num_leaves": 21,
    "learning_rate": 0.05,
    "feature_fraction": 0.9,
    "bagging_fraction": 0.8,
    "bagging_freq": 5,
    "max_depth": 12,
}<jupyter_output><empty_output><jupyter_text>Train the model<jupyter_code>gbm = lgb.train(
    params,
    lgb_train,
    num_boost_round=200,
    valid_sets=[lgb_train, lgb_eval],
    valid_names=["training", "validation"],
    callbacks=[neptune_callback],
)<jupyter_output><empty_output><jupyter_text>This cell do the model trainig and metadata logging to Neptune.  Log summary metadata to the same run under the "lgbm_summary" namespace<jupyter_code>y_pred = np.argmax(gbm.predict(X_test), axis=1)

# Log summary metadata to the same run under the "lgbm_summary" namespace
run["lgbm_summary"] = create_booster_summary(
    booster=gbm,
    log_trees=True,
    list_trees=[0, 1, 2, 3, 4],
    log_confusion_matrix=True,
    y_pred=y_pred,
    y_true=y_test,
)<jupyter_output><empty_output><jupyter_text>Stop logging**Warning:**Once you are done logging, you should stop tracking the run using the `stop()` method.This is needed only while logging from a notebook environment. While logging through a script, Neptune automatically stops tracking once the script has completed execution.<jupyter_code>run.stop()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10452695829094609, 'fraction_numerical': 0.009918616480162767, 'mean_word_length': 5.769363166953529, 'pattern_counts': {'":': 10, '<': 34, '<?xml version=': 0, '>': 34, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '218755', 'license': 'MIT', 'n_tokens_mistral': 1329, 'n_tokens_neox': 1284, 'n_words': 365, 'path': 'integrations-and-supported-tools/lightgbm/notebooks/Neptune_LightGBM.ipynb', 'repo_name': 'neptune-ai/examples', 'size': '3931'}