<jupyter_start><jupyter_text>The baseline model Finally we want to see how our baseline model keeps up. For this, we first need to upload the original dogImages (no style transferred images includes)<jupyter_code>import os

# this is the S3 data_dir name
data_dir = 'dogImages'

# set prefix, a descriptive name for a directory  
prefix = 'capstone-project-orig-data'

# upload data for the first time; otherwise use the second expression (because the S3 upload takes some time)
#train_location_orig_data = sagemaker_session.upload_data(os.path.join(data_dir), key_prefix=prefix)
train_location_orig_data = 's3://{}/{}/'.format(bucket, prefix)<jupyter_output><empty_output><jupyter_text>The estimator itself and the deployment is exactly the same. We only change the `source_dir`, that specifies our `train` and `predict` file. In case of our baseline we want to keep everything as simple as possible - that is why there is no advanced image augmenting included with the `imgaug` library. For more details have a look at `source_pytorch_style_transfer/train.py`<jupyter_code># your import and estimator code, here
from sagemaker.pytorch import PyTorch

output_path = 's3://{}/{}'.format(bucket, prefix)

# note: this PyTorch estimator is specified with code that only
# uses the style transferred pictures, but no imgaug preprocessing!
source_dir = 'source_pytorch_style_transfer'

# instantiate a pytorch estimator
estimator = PyTorch(entry_point='train.py',
                    source_dir=source_dir,
                    role=role,
                    framework_version='1.1.0',
                    train_instance_count=1,
                    #train_instance_type='ml.c4.xlarge',
                    train_instance_type='ml.p2.xlarge',
                    output_path=output_path,
                    sagemaker_session=sagemaker_session,
                    hyperparameters={
                        'epochs': 10
                    })


%%time
# Train your estimator on S3 training data
estimator.fit({'train': train_location_orig_data})
%%time

# importing PyTorchModel
from sagemaker.pytorch import PyTorchModel

print(estimator.model_data)

# Create a model from the trained estimator data
# And point to the prediction script
model = PyTorchModel(model_data=estimator.model_data,
                     role = role,
                     framework_version='1.1.0',
                     entry_point='predict.py',
                     source_dir=source_dir)
predictor = model.deploy(initial_instance_count=1, instance_type='ml.t2.large')
data_dir = 'dogImages'
print(os.path.join(data_dir, 'valid'))


# we create some dictionaries
image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x),
                                          data_transforms[x])
                  for x in ['train', 'valid', 'test']}
dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=5,
                                             shuffle=True, num_workers=4)
              for x in ['train', 'valid', 'test']}
dataset_sizes = {x: len(image_datasets[x]) for x in ['train', 'valid', 'test']}
testacc=test_acc(iter(dataloaders['test']), predictor)
print(testacc)
print(source_dir)
print(train_location_orig_data)<jupyter_output>source_pytorch_style_transfer
s3://sagemaker-eu-central-1-505649883860/capstone-project-orig-data/
{'fraction_non_alphanumeric': 0.0934971531315553, 'fraction_numerical': 0.010188792328438718, 'mean_word_length': 3.193467336683417, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '571440', 'license': 'MIT', 'n_tokens_mistral': 1004, 'n_tokens_neox': 965, 'n_words': 268, 'path': '04-dog-breed-sagemaker-deployment-vgg16_style-baseline.ipynb', 'repo_name': 'koljamaier/cnn-classifier', 'size': '3337'}