<jupyter_start><jupyter_text>Mortgage CrossValidation with GPU accelerating on XGBoostIn this notebook, we will show you how to levarage GPU to accelerate mortgage CrossValidation of XGBoost to find out the best model given a group of parameters. Import classesFirst we need load some common classes that both GPU version and CPU version will use:<jupyter_code>import ml.dmlc.xgboost4j.scala.spark.{XGBoostClassificationModel, XGBoostClassifier}

import org.apache.spark.sql.SparkSession
import org.apache.spark.ml.evaluation.MulticlassClassificationEvaluator
import org.apache.spark.ml.tuning.ParamGridBuilder
import org.apache.spark.sql.types.{FloatType, IntegerType, StructField, StructType}<jupyter_output><empty_output><jupyter_text>what is new to xgboost-spark users is **rapids.CrossValidator**<jupyter_code>import ml.dmlc.xgboost4j.scala.spark.rapids.CrossValidator<jupyter_output><empty_output><jupyter_text>Set dataset path<jupyter_code>val trainParquetPath="/data/mortgage/parquet/train"
val evalParquetPath="/data/mortgage/parquet/eval"<jupyter_output><empty_output><jupyter_text>Set the schema of the dataset<jupyter_code>val labelColName = "delinquency_12"
val schema = StructType(List(
    StructField("orig_channel", FloatType),
    StructField("first_home_buyer", FloatType),
    StructField("loan_purpose", FloatType),
    StructField("property_type", FloatType),
    StructField("occupancy_status", FloatType),
    StructField("property_state", FloatType),
    StructField("product_type", FloatType),
    StructField("relocation_mortgage_indicator", FloatType),
    StructField("seller_name", FloatType),
    StructField("mod_flag", FloatType),
    StructField("orig_interest_rate", FloatType),
    StructField("orig_upb", IntegerType),
    StructField("orig_loan_term", IntegerType),
    StructField("orig_ltv", FloatType),
    StructField("orig_cltv", FloatType),
    StructField("num_borrowers", FloatType),
    StructField("dti", FloatType),
    StructField("borrower_credit_score", FloatType),
    StructField("num_units", IntegerType),
    StructField("zip", IntegerType),
    StructField("mortgage_insurance_percent", FloatType),
    StructField("current_loan_delinquency_status", IntegerType),
    StructField("current_actual_upb", FloatType),
    StructField("interest_rate", FloatType),
    StructField("loan_age", FloatType),
    StructField("msa", FloatType),
    StructField("non_interest_bearing_upb", FloatType),
    StructField(labelColName, IntegerType)))<jupyter_output><empty_output><jupyter_text>Create a new spark session and load datawe must create a new spark session to continue all spark operations. It will also be used to initilize the `GpuDataReader` which is a data reader powered by GPU.NOTE: in this notebook, we have uploaded dependency jars when installing toree kernel. If we don't upload them at installation time, we can also upload in notebook by [%AddJar magic](https://toree.incubator.apache.org/docs/current/user/faq/). However, there's one restriction for `%AddJar`: the jar uploaded can only be available when `AddJar` is called after a new spark session is created. We must use it as below:```scalaimport org.apache.spark.sql.SparkSessionval spark = SparkSession.builder().appName("Taxi-GPU").getOrCreate%AddJar file:/data/libs/cudf-0.17-cuda10-1.jar%AddJar file:/data/libs/xgboost4j_3.0-1.3.0-0.1.0.jar%AddJar file:/data/libs/xgboost4j-spark_3.0-1.3.0-0.1.0.jar%AddJar file:/data/libs/rapids-4-spark_2.12-0.3.0.jar// ...```<jupyter_code>val spark = SparkSession.builder().appName("mortgage-gpu-cv").getOrCreate()
val trainDs = spark.read.parquet(trainParquetPath)<jupyter_output><empty_output><jupyter_text>Find out features to train<jupyter_code>val featureNames = schema.filter(_.name != labelColName).map(_.name)
val classifierParam = Map(
    "eta" -> 0.1,
    "gamma" -> 0.1,
    "missing" -> 0.0,
    "max_depth" -> 10,
    "max_leaves" -> 256,
    "grow_policy" -> "depthwise",
    "objective" -> "binary:logistic",
    "min_child_weight" -> 30,
    "lambda" -> 1,
    "scale_pos_weight" -> 2,
    "subsample" -> 1,
    "nthread" -> 1,
    "num_round" -> 100,
    "tree_method" -> "gpu_hist")<jupyter_output><empty_output><jupyter_text>Construct CrossValidator<jupyter_code>val classifier = new XGBoostClassifier(classifierParam)
    .setLabelCol(labelColName)
    .setFeaturesCols(featureNames)
val paramGrid = new ParamGridBuilder()
    .addGrid(classifier.maxDepth, Array(3, 10))
    .addGrid(classifier.eta, Array(0.2, 0.6))
    .build()
val evaluator = new MulticlassClassificationEvaluator().setLabelCol(labelColName)
val cv = new CrossValidator()
    .setEstimator(classifier)
    .setEvaluator(evaluator)
    .setEstimatorParamMaps(paramGrid)
    .setNumFolds(3)<jupyter_output><empty_output><jupyter_text>train with CrossValidator<jupyter_code>val model = cv.fit(trainDs).bestModel.asInstanceOf[XGBoostClassificationModel]<jupyter_output>Tracker started, with env={DMLC_NUM_SERVER=0, DMLC_TRACKER_URI=10.19.183.93, DMLC_TRACKER_PORT=9091, DMLC_NUM_WORKER=1}
Tracker started, with env={DMLC_NUM_SERVER=0, DMLC_TRACKER_URI=10.19.183.93, DMLC_TRACKER_PORT=9091, DMLC_NUM_WORKER=1}
Tracker started, with env={DMLC_NUM_SERVER=0, DMLC_TRACKER_URI=10.19.183.93, DMLC_TRACKER_PORT=9091, DMLC_NUM_WORKER=1}
Tracker started, with env={DMLC_NUM_SERVER=0, DMLC_TRACKER_URI=10.19.183.93, DMLC_TRACKER_PORT=9091, DMLC_NUM_WORKER=1}
Tracker started, with env={DMLC_NUM_SERVER=0, DMLC_TRACKER_URI=10.19.183.93, DMLC_TRACKER_PORT=9091, DMLC_NUM_WORKER=1}
Tracker started, with env={DMLC_NUM_SERVER=0, DMLC_TRACKER_URI=10.19.183.93, DMLC_TRACKER_PORT=9091, DMLC_NUM_WORKER=1}
Tracker started, with env={DMLC_NUM_SERVER=0, DMLC_TRACKER_URI=10.19.183.93, DMLC_TRACKER_PORT=9091, DMLC_NUM_WORKER=1}
Tracker started, with env={DMLC_NUM_SERVER=0, DMLC_TRACKER_URI=10.19.183.93, DMLC_TRACKER_PORT=9091, DMLC_NUM_WORKER=1}
Tracker started, with env={DMLC_NUM_SERV[...]<jupyter_text>tranform with best model trained by CrossValidator<jupyter_code>val transformDs = spark.read.parquet(evalParquetPath)
val df = model.transform(transformDs).cache()
df.drop(featureNames: _*).show(5)
val evaluator = new MulticlassClassificationEvaluator().setLabelCol(labelColName)
val accuracy = evaluator.evaluate(df)
spark.close()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11815150071462602, 'fraction_numerical': 0.02922026361759568, 'mean_word_length': 6.606280193236715, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 50, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 8, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '112826', 'license': 'Apache-2.0', 'n_tokens_mistral': 2350, 'n_tokens_neox': 2208, 'n_words': 401, 'path': 'examples/notebooks/scala/mortgage_gpu_crossvalidation.ipynb', 'repo_name': 'shotai/spark-xgboost-examples', 'size': '6297'}