<jupyter_start><jupyter_text>Huggingface Sagemaker-sdk extension example using `Trainer` class  Installs requirements if you haven´t already done it and sets up ipywidgets for datasets in sagemaker studio<jupyter_code>%%capture
!pip install -r ../requirements.txt --upgrade
%%capture
import os 
import IPython
if 'SAGEMAKER_TRAINING_MODULE' in os.environ:
    !conda install -c conda-forge ipywidgets -y
    IPython.Application.instance().kernel.do_shutdown(True) # has to restart kernel so changes are used<jupyter_output><empty_output><jupyter_text>Initializing Sagemaker Session with local AWS Profile From outside these notebooks, `get_execution_role()` will return an exception because it does not know what is the role name that SageMaker requires.To solve this issue, pass the IAM role name instead of using `get_execution_role()`.Therefore you have to create an IAM-Role with correct permission for sagemaker to start training jobs and download files from s3. Beware that you need s3 permission on bucket-level `"arn:aws:s3:::sagemaker-*"` and on object-level     `"arn:aws:s3:::sagemaker-*/*"`. You can read [here](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html) how to create a role with right permissions.<jupyter_code># local aws profile configured in ~/.aws/credentials
local_profile_name='hf-sm' # optional if you only have default configured

# role name for sagemaker -> needs the described permissions from above
role_name = "SageMakerRole"
import sagemaker
import os
try:
    sess = sagemaker.Session()
    role = sagemaker.get_execution_role()
except Exception:
    import boto3
    # creates a boto3 session using the local profile we defined
    if local_profile_name:
        os.environ['AWS_PROFILE'] = local_profile_name # setting env var bc local-mode cannot use boto3 session
        #bt3 = boto3.session.Session(profile_name=local_profile_name)
        #iam = bt3.client('iam')
        # create sagemaker session with boto3 session
        #sess = sagemaker.Session(boto_session=bt3)
    iam = boto3.client('iam')
    sess = sagemaker.Session()
    # get role arn
    role = iam.get_role(RoleName=role_name)['Role']['Arn']
    


print(role)<jupyter_output>Couldn't call 'get_role' to get Role ARN from role name philipp to get Role path.<jupyter_text>Sagemaker Session prints<jupyter_code>print(sess.list_s3_files(sess.default_bucket(),'datasets/')) # list objects in s3 under datsets/
print(sess.default_bucket()) # s3 bucketname
print(sess.boto_region_name) # aws region of sagemaker session<jupyter_output>['datasets/imdb/small/test/dataset.arrow', 'datasets/imdb/small/test/dataset_info.json', 'datasets/imdb/small/test/state.json', 'datasets/imdb/small/test/test_dataset.pt', 'datasets/imdb/small/train/dataset.arrow', 'datasets/imdb/small/train/dataset_info.json', 'datasets/imdb/small/train/state.json', 'datasets/imdb/small/training/train_dataset.pt', 'datasets/imdb/test/dataset.arrow', 'datasets/imdb/test/dataset_info.json', 'datasets/imdb/test/state.json', 'datasets/imdb/train/dataset.arrow', 'datasets/imdb/train/dataset_info.json', 'datasets/imdb/train/state.json']
sagemaker-eu-central-1-558105141721
eu-central-1<jupyter_text>ImportsSince we are using the `.py` module directly from `huggingface/` we have to adjust our `sys.path` to be able to import our estimator<jupyter_code>import sys, os

module_path = os.path.abspath(os.path.join('../../src'))
if module_path not in sys.path:
    sys.path.append(module_path)

%load_ext autoreload
%autoreload 2
from huggingface.estimator import HuggingFace<jupyter_output><empty_output><jupyter_text>Create an Estimator with an Experiment[Metric Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/training-metrics.html)To find a metric, SageMaker searches the logs that your algorithm emits and finds logs that match the regular expression that you specify for that metric. Defining Training Metrics (SageMaker Python SDK)Define the metrics that you want to send to CloudWatch by specifying a list of metric names and regular expressions as the metric_definitions argument when you initialize an Estimator object. For example, if you want to monitor both the train:error and validation:error metrics in CloudWatch, your Estimator initialization would look like the following:```pythonEstimator(    ...,    sagemaker_session = sm_sess,    tags = [{'Key': 'my-experiments', 'Value': 'demo2'}])estimator.fit(    ...,    experiment_config = {         "ExperimentName"        "TrialName" : demo_trial.trial_name,        "TrialComponentDisplayName" : "TrainingJob",    })```In the regex for the train:error metric defined above, the first part of the regex finds the exact text `"Train_error="`, and the expression `(.*?);` captures zero or more of any character until the first `;` semicolon character.For more information about training by using Amazon SageMaker Python SDK estimators, see https://github.com/aws/sagemaker-python-sdksagemaker-python-sdk-overview. Scriptshttps://github.com/huggingface/transformers/tree/master/examples/text-classification GLUE    export TASK_NAME=MRPC    python run_glue.py \      --model_name_or_path bert-base-cased \      --task_name $TASK_NAME \      --do_train \      --do_eval \      --max_seq_length 128 \      --per_device_train_batch_size 32 \      --learning_rate 2e-5 \      --num_train_epochs 3.0 \      --output_dir /tmp/$TASK_NAME/       XNLI          export XNLI_DIR=/path/to/XNLI    python run_xnli.py \      --model_name_or_path bert-base-multilingual-cased \      --language de \      --train_language en \      --do_train \      --do_eval \      --data_dir $XNLI_DIR \      --per_device_train_batch_size 32 \      --learning_rate 5e-5 \      --num_train_epochs 2.0 \      --max_seq_length 128 \      --output_dir /tmp/debug_xnli/ \      --save_steps -1  Local Estimator<jupyter_code>from huggingface.estimator import HuggingFace
import json

local_estimator = HuggingFace(entry_point='run_glue.py',
                            source_dir='../../transformers/examples/text-classification',
                            base_job_name='huggingface-sdk-extension',
                            instance_type='local',
                            instance_count=1,
                            role=role,
                            framework_version={'transformers':'4.1.1','datasets':'1.1.3'},
                            py_version='py3',
                            hyperparameters = {
                                'model_name_or_path': 'distilbert-base-cased',
                                'task_name':'MRPC',
                                'do_train': True,
                                'do_eval': True,
                                'max_seq_length':'128',
                                'per_device_train_batch_size':32,
                                'learning_rate':2e-5,
                                'num_train_epochs': 3.0
                            })
local_estimator.fit()<jupyter_output>Creating sl8nyno36v-algo-1-3upme ... 
[1BAttaching to sl8nyno36v-algo-1-3upme2mdone[0m
[36msl8nyno36v-algo-1-3upme |[0m 2020-12-30 16:05:11,627 sagemaker-training-toolkit INFO     Imported framework sagemaker_pytorch_container.training
[36msl8nyno36v-algo-1-3upme |[0m 2020-12-30 16:05:11,629 sagemaker-training-toolkit INFO     No GPUs detected (normal if no gpus installed)
[36msl8nyno36v-algo-1-3upme |[0m 2020-12-30 16:05:11,645 sagemaker_pytorch_container.training INFO     Block until all host DNS lookups succeed.
[36msl8nyno36v-algo-1-3upme |[0m 2020-12-30 16:05:11,648 sagemaker_pytorch_container.training INFO     Invoking user training script.
[36msl8nyno36v-algo-1-3upme |[0m 2020-12-30 16:05:11,678 botocore.credentials INFO     Found credentials in environment variables.
[36msl8nyno36v-algo-1-3upme |[0m 2020-12-30 16:05:11,912 sagemaker-training-toolkit INFO     Installing dependencies from requirements.txt:
[36msl8nyno36v-algo-1-3upme |[0m /opt/conda/bin/python -m [...]<jupyter_text>Sagemaker Estimator<jupyter_code>from huggingface.estimator import HuggingFace


huggingface_estimator = HuggingFace(entry_point='run_glue.py',
                            source_dir='../../transformers/examples/text-classification',
                            sagemaker_session=sess,
                            base_job_name='huggingface-sdk-extension',
                            instance_type='ml.p3.2xlarge',
                            instance_count=1,
                            role=role,
                            framework_version={'transformers':'4.1.1','datasets':'1.1.3'},
                            py_version='py3',
                            hyperparameters = {
                                'model_name_or_path': 'distilbert-base-cased',
                                'task_name':'MRPC',
                                'do_train': True,
                                'do_eval': True,
                                'max_seq_length':'128',
                                'per_device_train_batch_size':32,
                                'learning_rate':2e-5,
                                'num_train_epochs': 3.0
                            })
huggingface_estimator.fit()<jupyter_output>2020-12-31 08:22:11 Starting - Starting the training job...
2020-12-31 08:22:34 Starting - Launching requested ML instancesProfilerReport-1609402930: InProgress
......
2020-12-31 08:23:35 Starting - Preparing the instances for training......
2020-12-31 08:24:36 Downloading - Downloading input data
2020-12-31 08:24:36 Training - Downloading the training image.....................
2020-12-31 08:28:12 Training - Training image download completed. Training in progress..[34mbash: cannot set terminal process group (-1): Inappropriate ioctl for device[0m
[34mbash: no job control in this shell[0m
[34m2020-12-31 08:28:12,243 sagemaker-training-toolkit INFO     Imported framework sagemaker_pytorch_container.training[0m
[34m2020-12-31 08:28:12,266 sagemaker_pytorch_container.training INFO     Block until all host DNS lookups succeed.[0m
[34m2020-12-31 08:28:12,498 sagemaker_pytorch_container.training INFO     Invoking user training script.[0m
[34m2020-12-31 08:28:12,878 sagemaker-train[...]
{'fraction_non_alphanumeric': 0.10560281607509533, 'fraction_numerical': 0.04282780874156644, 'mean_word_length': 2.8978658536585367, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 22, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '631740', 'license': 'Apache-2.0', 'n_tokens_mistral': 3462, 'n_tokens_neox': 3125, 'n_words': 727, 'path': 'examples/06_transformers_existing_training_scripts/sagemaker-notebook.ipynb', 'repo_name': 'n1t0/sagemaker-sdk-huggingface', 'size': '10227'}