<jupyter_start><jupyter_text>Direct Marketing with Amazon SageMaker Autopilot------Kernel `Python 3 (Data Science)` works well with this notebook.  Contents1. [Upload the dataset to Amazon S3](Uploading)1. [Setting up the SageMaker Autopilot Job](Settingup)1. [Launching the SageMaker Autopilot Job](Launching)1. [Tracking Sagemaker Autopilot Job Progress](Tracking)1. [Results](Results)1. [Cleanup](Cleanup)  PrerequisitesPlease run prepareData.ipynb file before you continue the following steps Working with prepareDataNow we have the data we can start building our model. Let's set up the environment variables again in the next cell.<jupyter_code>import sagemaker
import boto3
from sagemaker import get_execution_role

region = boto3.Session().region_name

session = sagemaker.Session()
bucket = session.default_bucket()
prefix = 'sagemaker/autopilot-dm'

role = get_execution_role()

sm = boto3.Session().client(service_name='sagemaker',region_name=region)<jupyter_output><empty_output><jupyter_text>Pandas is a data manipulatiom library written for Python. It is very powerful and is first choice for many data scientists when it comes to data analysis in Python. So we will start with loading the data into a Pandas data frame and take a look at the data and try to udnerstand different columns.<jupyter_code>import pandas as pd

local_data_path = './bank-additional/bank-additional-full.csv'
data = pd.read_csv(local_data_path)
pd.set_option('display.max_columns', 500)     # Make sure we can see all of the columns
pd.set_option('display.max_rows', 10)         # Keep the output on one page
data<jupyter_output><empty_output><jupyter_text>Note that there are 20 features to help predict the target column 'y'.As explaind in the begining of the workshop, the first and most time consuming task for a data scientist is the data wrangling, e.g. clean up data format such as date fields, deal woth missing data, clean up outliers etc. Amazon SageMaker Autopilot takes care of preprocessing your data for you. You do not need to perform conventional data preprocessing techniques such as handling missing values, converting categorical features to numeric features, scaling data, and handling more complicated data types. Moreover, splitting the dataset into training and validation splits is not necessary. Autopilot takes care of this for you. We will split the test and training data. By training data we refer to training and validation data which are going to be used in the training phase. However, that split is done automatically by AutoML so we just call all of it training data in the following cell.<jupyter_code>train_data = data.sample(frac=0.8,random_state=200)

test_data = data.drop(train_data.index)

test_data_no_target = test_data.drop(columns=['y'])<jupyter_output><empty_output><jupyter_text>Upload the dataset to Amazon S3All the ML models have input and output channels. That is, you need to tell where the data is and where the training algorithm should save the final model. S3 is the common mechanism for this purpose. So we need to copy the test and training data to S3 in a .csv format for Amazon SageMaker training to use. Please run the following cell to achive this task.<jupyter_code>train_file = 'train_data.csv';
train_data.to_csv(train_file, index=False, header=True)
train_data_s3_path = session.upload_data(path=train_file, key_prefix=prefix + "/train")
print('Train data uploaded to: ' + train_data_s3_path + 'under bucket: ' + bucket)

test_file = 'test_data.csv';
test_data_no_target.to_csv(test_file, index=False, header=False)
test_data_s3_path = session.upload_data(path=test_file, key_prefix=prefix + "/test")
print('Test data uploaded to: ' + test_data_s3_path + 'under bucket: ' + bucket)<jupyter_output><empty_output><jupyter_text>Setting up the SageMaker Autopilot JobAfter uploading the dataset to Amazon S3, you can invoke Autopilot to find the best ML pipeline to train a model on this dataset. The required inputs for invoking a Autopilot job are:* Amazon S3 location for input dataset and for all output artefacts* Name of the column of the dataset you want to predict (`y` in this case) * An IAM roleCurrently Autopilot supports only tabular datasets in CSV format. Either all files should have a header row, or the first file of the dataset, when sorted in alphabetical/lexical order, is expected to have a header row. As discussed earlier we will call these parameters as input and output data configs, ie. input/output channels.<jupyter_code>input_data_config = [{
      'DataSource': {
        'S3DataSource': {
          'S3DataType': 'S3Prefix',
          'S3Uri': 's3://{}/{}/train'.format(bucket,prefix)
        }
      },
      'TargetAttributeName': 'y'
    }
  ]

output_data_config = {
    'S3OutputPath': 's3://{}/{}/output'.format(bucket,prefix)
  }<jupyter_output><empty_output><jupyter_text>You can also specify the type of problem you want to solve with your dataset (`Regression, MulticlassClassification, BinaryClassification`). In case you are not sure, SageMaker Autopilot will infer the problem type based on statistics of the target column (the column you want to predict). For example if the target column has only 2 values as 0 and 1 this is a Binayr classification. If you have more than 2 values then it is MulticlassClassification. Finally, if you have a double precision values e.g. 1.23 in your target column than your problme becomes a regression. But again, you do not need to know these differnences since AutoML is smrt enough to figure it out from the data.You have the option to limit the running time of a SageMaker Autopilot job by providing either the maximum number of pipeline evaluations or candidates (one pipeline evaluation is called a `Candidate` because it generates a candidate model) or providing the total time allocated for the overall Autopilot job. Under default settings, this job takes about four hours to run. This varies between runs because of the nature of the exploratory process Autopilot uses to find optimal training parameters.  Launching the SageMaker Autopilot JobYou can now launch the Autopilot job by calling the `create_auto_ml_job` API.<jupyter_code>from time import gmtime, strftime, sleep
timestamp_suffix = strftime('%d-%H-%M-%S', gmtime())

auto_ml_job_name = 'automl-banking-' + timestamp_suffix
print('AutoMLJobName: ' + auto_ml_job_name)

sm.create_auto_ml_job(AutoMLJobName=auto_ml_job_name,
                      InputDataConfig=input_data_config,
                      OutputDataConfig=output_data_config,
                      RoleArn=role)<jupyter_output><empty_output><jupyter_text>Tracking SageMaker Autopilot job progressSageMaker Autopilot job consists of the following high-level steps : * Analyzing Data, where the dataset is analyzed and Autopilot comes up with a list of ML pipelines that should be tried out on the dataset. The dataset is also split into train and validation sets.* Feature Engineering, where Autopilot performs feature transformation on individual features of the dataset as well as at an aggregate level.* Model Tuning, where the top performing pipeline is selected along with the optimal hyperparameters for the training algorithm (the last stage of the pipeline). CHECK POINT: this will take around 15 minutes. You can continue to [third part of the workshop](https://github.com/daghanacay/ai-with-sagemaker-studio/blob/main/Step3-RunExistingModel/README.md) while this job is running. We will do this to save time and we wil jump back and forth between sagemaker studio notebooks.<jupyter_code>print ('JobStatus - Secondary Status')
print('------------------------------')


describe_response = sm.describe_auto_ml_job(AutoMLJobName=auto_ml_job_name)
print (describe_response['AutoMLJobStatus'] + " - " + describe_response['AutoMLJobSecondaryStatus'])
job_run_status = describe_response['AutoMLJobStatus']
    
while job_run_status not in ('Failed', 'Completed', 'Stopped'):
    describe_response = sm.describe_auto_ml_job(AutoMLJobName=auto_ml_job_name)
    job_run_status = describe_response['AutoMLJobStatus']
    
    print (describe_response['AutoMLJobStatus'] + " - " + describe_response['AutoMLJobSecondaryStatus'])
    sleep(30)<jupyter_output><empty_output><jupyter_text>ResultsWhile we were working on the third part of the workshop AutoML - Cleaned up the data- Detect the problem type, eg. Regression, MulticlassClassification, BinaryClassification- run over training data and find best performing model and stred into multiple candidate modelsWe can see the models using the describe_auto_ml_job API. We will look up the best candidate out of all the models selected by the SageMaker Autopilot job. Please run the follwing cell to get the information about the best performing model.<jupyter_code>best_candidate = sm.describe_auto_ml_job(AutoMLJobName=auto_ml_job_name)['BestCandidate']
best_candidate_name = best_candidate['CandidateName']
print(best_candidate)
print('\n')
print("CandidateName: " + best_candidate_name)
print("FinalAutoMLJobObjectiveMetricName: " + best_candidate['FinalAutoMLJobObjectiveMetric']['MetricName'])
print("FinalAutoMLJobObjectiveMetricValue: " + str(best_candidate['FinalAutoMLJobObjectiveMetric']['Value']))<jupyter_output><empty_output><jupyter_text>Perform batch inference using the best candidateNow that you have successfully completed the SageMaker Autopilot job on the dataset, create a model from any of the candidates by using [Inference Pipelines](https://docs.aws.amazon.com/sagemaker/latest/dg/inference-pipelines.html).<jupyter_code>model_name = 'automl-banking-model-' + timestamp_suffix

model = sm.create_model(Containers=best_candidate['InferenceContainers'],
                            ModelName=model_name,
                            ExecutionRoleArn=role)

print('Model ARN corresponding to the best candidate is : {}'.format(model['ModelArn']))<jupyter_output><empty_output><jupyter_text>You can use batch inference by using Amazon SageMaker batch transform. For now run the next two cells and come back to here to continue reading. The reason is that, the batch job will take around 5minutes and we can inspect what happened while the job is running.Now lets go to AWS sagemaker console and see the model we have created. Under Sagemaker console https://console.aws.amazon.com/sagemaker/home?region=us-east-1/landing go to "Inference -> Models". You should see automl-banking-model-' + timestamp_suffix. If you click on this model you will see where the model is stored in S3 bucket.If you have used sagemaker before you can use this model for creating API endpoints from "Inference-> Endpoint configurations" and later with "Inference -> endpoints". However, we will not explore those in this step. We will look into that in [third part of the workshop](https://github.com/daghanacay/ai-with-sagemaker-studio/blob/main/Step3-RunExistingModel/README.md).The other thing that you can inspect is the transform job. You can see this job in the sagemake console "Inference -> Batch Processing jobs"As we have started our batch job earlier, it should have completed by now.<jupyter_code>transform_job_name = 'automl-banking-transform-' + timestamp_suffix

transform_input = {
        'DataSource': {
            'S3DataSource': {
                'S3DataType': 'S3Prefix',
                'S3Uri': test_data_s3_path
            }
        },
        'ContentType': 'text/csv',
        'CompressionType': 'None',
        'SplitType': 'Line'
    }

transform_output = {
        'S3OutputPath': 's3://{}/{}/inference-results'.format(bucket,prefix),
    }

transform_resources = {
        'InstanceType': 'ml.m5.4xlarge',
        'InstanceCount': 1
    }

sm.create_transform_job(TransformJobName = transform_job_name,
                        ModelName = model_name,
                        TransformInput = transform_input,
                        TransformOutput = transform_output,
                        TransformResources = transform_resources
)<jupyter_output><empty_output><jupyter_text>Watch the transform job for completion.<jupyter_code>print ('JobStatus')
print('----------')


describe_response = sm.describe_transform_job(TransformJobName = transform_job_name)
job_run_status = describe_response['TransformJobStatus']
print (job_run_status)

while job_run_status not in ('Failed', 'Completed', 'Stopped'):
    describe_response = sm.describe_transform_job(TransformJobName = transform_job_name)
    job_run_status = describe_response['TransformJobStatus']
    print (job_run_status)
    sleep(30)<jupyter_output><empty_output><jupyter_text>As we have specified the output of the transform job to be "inference-results/test_data.csv.out" in our output bucket you can go and inspect the results there. However, below is the convenient way of copying back to Sagemaker Studio.In order to make sense of the test results please double click on the "test_data.csv" and compare the results based on the input and the predicted output whether a customer will enrol for a term deposit<jupyter_code>s3_output_key = '{}/inference-results/test_data.csv.out'.format(prefix);
local_inference_results_path = 'inference_results.csv'

s3 = boto3.resource('s3')
inference_results_bucket = s3.Bucket(session.default_bucket())

inference_results_bucket.download_file(s3_output_key, local_inference_results_path);

data = pd.read_csv(local_inference_results_path, sep=';')
pd.set_option('display.max_rows', 10)         # Keep the output on one page
data<jupyter_output><empty_output><jupyter_text>Candidate Generation Notebook    Sagemaker AutoPilot also auto-generates a Candidate Definitions notebook. This notebook can be used to interactively step through the various steps taken by the Sagemaker Autopilot to arrive at the best candidate. This notebook can also be used to override various runtime parameters like parallelism, hardware used, algorithms explored, feature extraction scripts and more.    The notebook can be downloaded from the following Amazon S3 location:<jupyter_code>sm.describe_auto_ml_job(AutoMLJobName=auto_ml_job_name)['AutoMLJobArtifacts']['CandidateDefinitionNotebookLocation']<jupyter_output><empty_output><jupyter_text>Data Exploration NotebookSagemaker Autopilot also auto-generates a Data Exploration notebook, which can be downloaded from the following Amazon S3 location:<jupyter_code>sm.describe_auto_ml_job(AutoMLJobName=auto_ml_job_name)['AutoMLJobArtifacts']['DataExplorationNotebookLocation']

I suggest you read through these files to get a deeper understanding of the inner workings of the AutoML. Also you can use them to change different parts of the AI/ML pipeline.

Please go back to [third part of the workshop](https://github.com/daghanacay/ai-with-sagemaker-studio/blob/main/Step3-RunExistingModel/README.md)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07433664168523395, 'fraction_numerical': 0.005266356086692323, 'mean_word_length': 5.0040535062829345, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 61, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '307075', 'license': 'Apache-2.0', 'n_tokens_mistral': 4180, 'n_tokens_neox': 4022, 'n_words': 1540, 'path': 'Step2-RunAutoML/src/createModelandRunInference.ipynb', 'repo_name': 'kukielp/ai-with-sagemaker-studio', 'size': '14811'}