Q:

Where do I store my model's training data, artifacts, etc?

I'm trying to build and push a custom ML model with docker to Amazon SageMaker. I know things are supposed to follow the general structure of being in opt/ml. But there's no such bucket in Amazon S3??? Am I supposed to create this directory within my container before I build and push the image to AWS? I just have no idea where to put my training data, etc.

A:

SageMaker is automating the deployment of the Docker image with your code using the convention of channel->local-folder. Everything that you define with a channel in your input data configuration, will be copied to the local Docker file system under /opt/ml/ folder, using the name of the channel as the name of the sub-folder.
{
"train" : {"ContentType":  "trainingContentType", 
           "TrainingInputMode": "File", 
           "S3DistributionType": "FullyReplicated", 
           "RecordWrapperType": "None"},
"evaluation" : {"ContentType":  "evalContentType", 
                "TrainingInputMode": "File", 
                "S3DistributionType": "FullyReplicated", 
                "RecordWrapperType": "None"},
"validation" : {"TrainingInputMode": "File", 
                "S3DistributionType": "FullyReplicated", 
                "RecordWrapperType": "None"}
} 

to:
/opt/ml/input/data/training
/opt/ml/input/data/validation
/opt/ml/input/data/testing

