<jupyter_start><jupyter_text>Decision Forest Classification and Regression with Intel® Data Analytics Acceleration Library in Amazon SageMaker IntroductionIntel® Data Analytics Acceleration Library (Intel® DAAL) is the library of Intel® architecture optimized building blocks covering all stages of data analytics: data acquisition from a data source, preprocessing, transformation, data mining, modeling, validation, and decision making. One of its algorithms is Decision Forest.The library provides decision forest classification and regression algorithms based on an ensemble of tree-structured classifiers (decision trees) built using the general technique of bootstrap aggregation (bagging) and random choice of features. Decision tree is a binary tree graph. Its internal (split) nodes represent a decision function used to select the following (child) node at the prediction stage. Its leaf (terminal) nodes represent the corresponding response values, which are the result of the prediction from the tree.Intel® DAAL developer guide: https://software.intel.com/en-us/daal-programming-guideIntel® DAAL documentation for Decision Forest: https://software.intel.com/en-us/daal-programming-guide-decision-forest  Decision Forest Usage with SageMaker EstimatorFirstly, you need to import SageMaker package, get execution role and create session.<jupyter_code>import sagemaker

role = sagemaker.get_execution_role()
sess = sagemaker.Session()<jupyter_output><empty_output><jupyter_text>Secondly, you can specify parameters of Decision Forest. Hyperparameters            Parameter name        Type        Default value        Description                nClasses        int        3        Number of classes in data (only for classification)                fptype        str        "double"        The floating-point type that the algorithm uses for intermediate computations. Can be "float" or "double"                method        str        "defaultDense"        The only training method supported so far is the default dense method                nTrees        int        100        The number of trees in the forest                observationsPerTreeFraction        int        1        Fraction of the training set S used to form the bootstrap set for a single tree training, observationsPerTreeFraction in (0, 1]. The observations are sampled randomly with replacement                featuresPerNode        int        0        The number of features tried as possible splits per node. If the parameter is set to 0, the library uses the square root of the number of features for classification and (the number of features)/3 for regression                maxTreeDepth        int        0        Maximal tree depth. Default is 0 (unlimited).                minObservationsInLeafNode        int        1 for classification, 5 for regression        The number of neighbors                seed        int        777        The seed for random number generator, which is used to choose the bootstrap set, split features in every split node in a tree, and generate permutation required in computations of MDA variable importance                impurityThreshold        float        0        The threshold value used as stopping criteria: if the impurity value in the node is smaller than the threshold, the node is not split anymore                varImportance        str        None        The variable importance computation mode. Possible values:none – variable importance is not calculatedMDI - Mean Decrease of Impurity, also known as the Gini importance or Mean Decrease GiniMDA_Raw - Mean Decrease of Accuracy (permutation importance)MDA_Scaled - the MDA_Raw value scaled by its standard deviation                resultsToCompute        str        "computeOutOfBagError|computeOutOfBagErrorPerObservation"        Provide one of the following values to request a single characteristic or use bitwise OR to request a combination of the characteristics:computeOutOfBagError, computeOutOfBagErrorPerObservation                memorySavingMode        bool        False        If True, memory saving mode is enabled                bootstrap        bool        False for classification, True for regression        If True, bootstrap is enabled    Example of hyperparameters dictionary:<jupyter_code>decision_forest_params = {
    "nClasses": 3,
    "fptype":"double",
    "method":"defaultDense",
    "nTrees":"100",
    "observationsPerTreeFraction":"1",
    "featuresPerNode":"0",
    "maxTreeDepth":"0",
    "minObservationsInLeafNode":"1",
    "seed":"777",
    "impurityThreshold":"0",
    "varImportance":"None",
    "resultsToCompute":"0",
    "memorySavingMode":"False",
    "bootstrap":"False",
    "distributed":"False"
}<jupyter_output><empty_output><jupyter_text>Then, you need to create SageMaker Estimator instance with following parameters:            Parameter name        Description                image_name        The container image to use for training                role        An AWS IAM role. The SageMaker training jobs and APIs that create SageMaker endpoints use this role to access training data and models                train_instance_count        Number of Amazon EC2 instances to use for training. Should be 1, because it is not distributed version of algorithm                train_instance_type        Type of EC2 instance to use for training. See available types on Amazon Marketplace page of algorithm                input_mode        The input mode that the algorithm supports. May be "File" or "Pipe"                output_path        S3 location for saving the trainig result (model artifacts and output files)                sagemaker_session        Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed                hyperparameters        Dictionary containing the hyperparameters to initialize this estimator with    Full SageMaker Estimator documentation: https://sagemaker.readthedocs.io/en/latest/estimators.html<jupyter_code>daal_decision_forest_arn = "<algorithm-arn>" # you can find it on algorithm page in your subscriptions

daal_decision_forest = sagemaker.algorithm.AlgorithmEstimator(
    algorithm_arn=daal_decision_forest_arn,
    role=role,
    base_job_name="<base-job-name>",
    train_instance_count=1,
    train_instance_type='ml.m4.xlarge',
    input_mode="File",
    output_path="s3://<bucket-name>/<output-path>",
    sagemaker_session=sess,
    hyperparameters=decision_forest_params
)<jupyter_output><empty_output><jupyter_text>Training stageOn training stage, Decision Forest algorithm consume input data from S3 location.This container supports only .csv ("comma-separated values") files.<jupyter_code>daal_decision_forest.fit({"training": "s3://<bucket-name>/<training-data-path>"})<jupyter_output><empty_output><jupyter_text>Real-time predictionFirstly, you need to deploy SageMaker endpoint that consumes data.<jupyter_code>predictor = daal_decision_forest.deploy(1, "ml.m4.xlarge", serializer=sagemaker.predictor.csv_serializer)<jupyter_output><empty_output><jupyter_text>Secondly, you should pass data as numpy array to predictor instance and get transformed data as space-separated values.In this example we are passing random data, but you can use any numpy 2D array<jupyter_code>import numpy as np

predict_data = np.random.random(size=(10,10))
print(predictor.predict(predict_data).decode("utf-8"))<jupyter_output><empty_output><jupyter_text>Don't forget to delete endpoint if you don't need it anymore.<jupyter_code>sess.delete_endpoint(predictor.endpoint)<jupyter_output><empty_output><jupyter_text>Batch transform jobIf you don't need real-time prediction, you can use transform job. It uses saved model, compute transformed data one time and saves it in specified or auto-generated output path.More about transform jobs: https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-batch.htmlTransformer API: https://sagemaker.readthedocs.io/en/latest/transformer.html<jupyter_code>transformer = daal_decision_forest.transformer(1, 'ml.m4.xlarge')
transformer.transform("s3://<bucket-name>/<prediction-data-path>", content_type='text/csv')
transformer.wait()
print(transformer.output_path)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06023512301539207, 'fraction_numerical': 0.006181068961338019, 'mean_word_length': 3.6516347237880495, 'pattern_counts': {'":': 16, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '259987', 'license': 'MIT', 'n_tokens_mistral': 2108, 'n_tokens_neox': 1975, 'n_words': 793, 'path': 'jupyter/intel-daal/decision-forest-algorithm.ipynb', 'repo_name': 'Alexsandruss/sagemaker-examples', 'size': '8251'}