<jupyter_start><jupyter_text>Hyperparameter Tuning with Amazon SageMaker for Molecular Property Prediction  Contents1. [Background](Background)  2. [Setup](Setup)  3. [Code](Code)  4. [Tune](Tune)  5. [Wrap-up](Wrap-up)   BackgroundThis example notebook demonstrates a graph-based molecular property prediction model with automatic hyperparameter tuning. The implementation is based on DGL and PyTorch. To find the best hyperparameters, it leverages SageMaker to kick off multiple training jobs with different hyperparameter combinations. In this example, we'll use the [SageMaker Python SDK](https://github.com/aws/sagemaker-python-sdk) to create a hyperparameter tuning job.  SetupThis notebook was created and tested on an ml.p3.2xlarge notebook instance.We assume you have went through the `pytorch-gcn-tox21.ipynb` example and you have your account under your ECR specified by \{account\}.dkr.ecr.\{region\}.amazonaws.com/sagemaker-dgl-pytorch-gcn-tox21:latest.Let's start by specifying: * The S3 bucket and prefix that you want to use for training and model data. This should be within the same region as the notebook instance, training, and hosting. * The IAM role arn you are going to use to give training and hosting access to your data. See the documentation for more details on creating these. Note that if a role is not associated with the current notebook instance, or more than one role is required for training and/or hosting, you should replace sagemaker.get_execution_role() with the appropriate full IAM role arn string(s).<jupyter_code>import sagemaker

from sagemaker import get_execution_role
from sagemaker.session import Session

# Setup session
sess = sagemaker.Session()

# S3 bucket for saving code and model artifacts.
# Feel free to specify a different bucket here if you wish.
bucket = sess.default_bucket()

# Location to put your custom code.
custom_code_upload_location = 'customcode'

# IAM execution role that gives SageMaker access to resources in your AWS account.
# We can use the SageMaker Python SDK to get the role from our notebook environment. 
role = get_execution_role()<jupyter_output><empty_output><jupyter_text>CodeTo run docker containers with SageMaker, we need to provide a python script for the container to run. In this example, `main.py` provides all the code we need for training a SageMaker model.<jupyter_code>!cat main.py<jupyter_output><empty_output><jupyter_text>Tune<jupyter_code># Set target dgl-docker name
docker_name='sagemaker-dgl-pytorch-gcn-tox21'

CODE_PATH = 'main.py'
code_location = sess.upload_data(CODE_PATH, bucket=bucket, key_prefix=custom_code_upload_location)

account = sess.boto_session.client('sts').get_caller_identity()['Account']
region = sess.boto_session.region_name
image = '{}.dkr.ecr.{}.amazonaws.com/{}:latest'.format(account, region, docker_name)

estimator = sagemaker.estimator.Estimator(image,
                                          role, 
                                          train_instance_count=1, 
                                          train_instance_type='ml.p3.2xlarge',
                                          hyperparameters={'entrypoint': CODE_PATH},
                                          sagemaker_session=sess)<jupyter_output><empty_output><jupyter_text>Once we've defined our estimator, we can specify the hyperparameters we'd like to tune and their possible values. Depending on the type of possible values, the hyperparameters can be divided into three classes:* **Categorical**: Its possible values form a discrete set and is represented by `CategoricalParameter(list)`.* **Continuous**: It can take any real number within an interval `[min, max]` and is represented by `ContinuousParameter(min, max)`.* **Integer**: It can take any integer value within an interval `[min, max]` and is represented by `IntegerParameter(min, max)`.Note that it's almost always better to specify a value as the least restrictive type. For example, `ContinuousParameter(0.01, 0.2)` is less restrictive than `CategoricalParameter([0.01, 0.1, 0.15, 0.2])`.<jupyter_code>from sagemaker.tuner import IntegerParameter, CategoricalParameter, ContinuousParameter

hyper_ranges = {'lr': ContinuousParameter(1e-4, 1e-2),
                'patience': IntegerParameter(5, 30),
                'n_hidden': CategoricalParameter([32, 64, 128])}<jupyter_output><empty_output><jupyter_text>Next we'll specify the objective metric that we'd like to tune and its definition. This includes the regular expression (Regex) needed to extract that metric from the CloudWatch logs of our training job.<jupyter_code>objective_name = 'Validation_roc_auc'
metric_definitions = [{'Name': objective_name,
                       'Regex': 'Best validation score ([0-9\\.]+)'}]<jupyter_output><empty_output><jupyter_text>Now, we'll create a `HyperparameterTuner` object, which we pass: * The training estimator we created above * Our hyperparameter ranges * Objective metric name and definition * Number of training jobs to run in total and how many training jobs should be run simultaneously. More parallel jobs will finish tuning sooner, but may sacrifice accuracy. We recommend you set the parallel jobs value to less than 10% of the total number of training jobs (we'll set it higher just for this example to keep it short). * Whether we should maximize or minimize our objective metric (we haven't specified here since it defaults to 'Maximize', which is what we want for validation roc-auc)<jupyter_code>from sagemaker.tuner import HyperparameterTuner

tuner = HyperparameterTuner(estimator,
                            objective_name,
                            hyper_ranges,
                            metric_definitions,
                            max_jobs=6,
                            max_parallel_jobs=2)<jupyter_output><empty_output><jupyter_text>Finally, we can start our tuning job by calling `.fit()`.<jupyter_code>tuner.fit(inputs={'training-code': code_location})<jupyter_output><empty_output><jupyter_text>Let's just run a quick check of the hyperparameter tuning jobs status to make sure it started successfully and is InProgress.<jupyter_code>import boto3

boto3.client('sagemaker').describe_hyper_parameter_tuning_job(
    HyperParameterTuningJobName=tuner.latest_tuning_job.job_name)['HyperParameterTuningJobStatus']<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07658367475575166, 'fraction_numerical': 0.00866687677277025, 'mean_word_length': 4.202459016393442, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '616047', 'license': 'Apache-2.0', 'n_tokens_mistral': 1781, 'n_tokens_neox': 1692, 'n_words': 645, 'path': 'sagemaker-python-sdk/dgl_gcn_tox21/pytorch-gcn-tox21-hypertune.ipynb', 'repo_name': 'BarclayII/amazon-sagemaker-examples', 'size': '6346'}