<jupyter_start><jupyter_text>Upload Data to S3<jupyter_code># Specify your bucket name
bucket_name = 'chandra-ml-sagemaker'

training_folder = r'bikerental/training/'
validation_folder = r'bikerental/validation/'
test_folder = r'bikerental/test/'

s3_model_output_location = r's3://{0}/bikerental/model'.format(bucket_name)
s3_training_file_location = r's3://{0}/{1}'.format(bucket_name,training_folder)
s3_validation_file_location = r's3://{0}/{1}'.format(bucket_name,validation_folder)
s3_test_file_location = r's3://{0}/{1}'.format(bucket_name,test_folder)
print(s3_model_output_location)
print(s3_training_file_location)
print(s3_validation_file_location)
print(s3_test_file_location)
# Write and Reading from S3 is just as easy
# files are referred as objects in S3.  
# file name is referred as key name in S3

# File stored in S3 is automatically replicated across 3 different availability zones 
# in the region where the bucket was created.

# http://boto3.readthedocs.io/en/latest/guide/s3.html
def write_to_s3(filename, bucket, key):
    with open(filename,'rb') as f: # Read in binary mode
        return boto3.Session().resource('s3').Bucket(bucket).Object(key).upload_fileobj(f)
write_to_s3('bike_train.csv', 
            bucket_name,
            training_folder + 'bike_train.csv')

write_to_s3('bike_validation.csv',
            bucket_name,
            validation_folder + 'bike_validation.csv')

write_to_s3('bike_test.csv',
            bucket_name,
            test_folder + 'bike_test.csv')<jupyter_output><empty_output><jupyter_text>Training Algorithm Docker Image SageMaker maintains a separate image for algorithm and regionhttps://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html<jupyter_code># Establish a session with AWS
sess = sagemaker.Session()
role = get_execution_role()
# This role contains the permissions needed to train, deploy models
# SageMaker Service is trusted to assume this role
print(role)<jupyter_output><empty_output><jupyter_text>In the past, we had to maintain the algorithm containers mapping:containers = {'us-west-2': '433757028032.dkr.ecr.us-west-2.amazonaws.com/xgboost:latest',              'us-east-1': '811284229777.dkr.ecr.us-east-1.amazonaws.com/xgboost:latest',              'us-east-2': '825641698319.dkr.ecr.us-east-2.amazonaws.com/xgboost:latest',              'eu-west-1': '685385470294.dkr.ecr.eu-west-1.amazonaws.com/xgboost:latest'}Reference:https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html<jupyter_code># Sagemaker API now maintains the algorithm container mapping for us
# Specify the region, algorithm and version
container = sagemaker.amazon.amazon_estimator.get_image_uri(
    sess.boto_region_name,
    "xgboost", 
    "latest")

print('Using SageMaker XGBoost container:\n{} ({})'.format(container, sess.boto_region_name))<jupyter_output><empty_output><jupyter_text>Build Model<jupyter_code># Configure the training job
# Specify type and number of instances to use
# S3 location where final artifacts needs to be stored

#   Reference: http://sagemaker.readthedocs.io/en/latest/estimators.html

estimator = sagemaker.estimator.Estimator(
    container,
    role, 
    train_instance_count=1, 
    train_instance_type='ml.m4.xlarge',
    output_path=s3_model_output_location,
    sagemaker_session=sess,
    base_job_name ='xgboost-bikerental-v1')
# Specify hyper parameters that appropriate for the training algorithm
# XGBoost Training Parameter Reference: 
#   https://github.com/dmlc/xgboost/blob/master/doc/parameter.md

# max_depth=5,eta=0.1,subsample=0.7,num_round=150
estimator.set_hyperparameters(max_depth=5,
                              objective="reg:linear",
                              eta=0.1,
                              num_round=150)
estimator.hyperparameters()<jupyter_output><empty_output><jupyter_text>Specify Training Data Location and Optionally, Validation Data Location<jupyter_code># content type can be libsvm or csv for XGBoost
training_input_config = sagemaker.session.s3_input(
    s3_data=s3_training_file_location,
    content_type='csv',
    s3_data_type='S3Prefix')

validation_input_config = sagemaker.session.s3_input(
    s3_data=s3_validation_file_location,
    content_type='csv',
    s3_data_type='S3Prefix'
)

data_channels = {'train': training_input_config, 'validation': validation_input_config}
print(training_input_config.config)
print(validation_input_config.config)<jupyter_output><empty_output><jupyter_text>Train the model<jupyter_code># XGBoost supports "train", "validation" channels
# Reference: Supported channels by algorithm
#   https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html
estimator.fit(data_channels)<jupyter_output><empty_output><jupyter_text>Deploy Model<jupyter_code># Ref: http://sagemaker.readthedocs.io/en/latest/estimators.html
predictor = estimator.deploy(initial_instance_count=1,
                             instance_type='ml.m4.xlarge',
                             endpoint_name = 'xgboost-bikerental-v1')<jupyter_output><empty_output><jupyter_text>Run Predictions<jupyter_code>from sagemaker.predictor import csv_serializer, json_deserializer

predictor.content_type = 'text/csv'
predictor.serializer = csv_serializer
predictor.deserializer = None
predictor.predict([[3,0,1,2,28.7,33.335,79,12.998,2011,7,7,3]])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11000184877056757, 'fraction_numerical': 0.027176927343316695, 'mean_word_length': 4.958149779735683, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '176009', 'license': 'Apache-2.0', 'n_tokens_mistral': 1913, 'n_tokens_neox': 1835, 'n_words': 307, 'path': '5 Xgboost/BikeSharingRegression/sdk1.7/xgboost_cloud_training_template.ipynb', 'repo_name': 'jaypeeml/AWSSagemaker', 'size': '5409'}