<jupyter_start><jupyter_text>Tic Tac Toe @Edge with SageMaker Neo + Tensorflow and Reinforcement Learning (RLLib)**SageMaker Studio Kernel**: Data ScienceThis exercise is based on [TicTacToe with RL Tensorflow](https://github.com/aws-samples/ml-edge-getting-started/blob/main/examples/01_TicTacToe_RL_Tensorflow/01_TicTacToeRLLIB.ipynb). But, instead of training an agent, we'll just download a pre-trained 'mediocre' agent (a TF model) and create a board to play Tic Tac Toe against it. Let's do it!In this exercise you'll:   - Get a pre-trained model/agent that plays TicTacToe   - Prepare the model to compile it with Neo   - Compile the model for the target: **X86_64**   - Get the optimized model and run a simple game   - Have fun!  install dependencies<jupyter_code>!pip install dlr==1.8.0<jupyter_output><empty_output><jupyter_text>1) Get the model, create a package and upload it to S3<jupyter_code>import tarfile
import sagemaker
import os
import urllib.request

model_url = 'https://spock.cloud/models/tic-tac-toe-tf.tar.gz'
urllib.request.urlretrieve(model_url, 'model.tar.gz')

sagemaker_session = sagemaker.Session()
model_name='tic-tac-toe-tf'

s3_uri = sagemaker_session.upload_data('model.tar.gz', key_prefix=f'{model_name}/model')
print(s3_uri)<jupyter_output><empty_output><jupyter_text>2) Compile the model with SageMaker Neo (X86_64)<jupyter_code>import time
import boto3
import sagemaker

role = sagemaker.get_execution_role()
sm_client = boto3.client('sagemaker')
compilation_job_name = f'{model_name}-tensorflow-{int(time.time()*1000)}'
sm_client.create_compilation_job(
    CompilationJobName=compilation_job_name,
    RoleArn=role,
    InputConfig={
        'S3Uri': s3_uri,
        'DataInputConfig': '{"agent_x/observations": [1,10]}',
        'Framework': 'TENSORFLOW'
    },
    OutputConfig={
        'S3OutputLocation': f's3://{sagemaker_session.default_bucket()}/{model_name}-tensorflow/optimized/',
        'TargetPlatform': { 
            'Os': 'LINUX', 
            'Arch': 'X86_64',
        },
    },
    StoppingCondition={ 'MaxRuntimeInSeconds': 900 }
)
while True:
    resp = sm_client.describe_compilation_job(CompilationJobName=compilation_job_name)    
    if resp['CompilationJobStatus'] in ['STARTING', 'INPROGRESS']:
        print('Running...')
    else:
        print(resp['CompilationJobStatus'], compilation_job_name)
        break
    time.sleep(5)<jupyter_output><empty_output><jupyter_text>3) Download the compiled model<jupyter_code>output_model_path = f's3://{sagemaker_session.default_bucket()}/{model_name}-tensorflow/optimized/model-LINUX_X86_64.tar.gz'
!aws s3 cp $output_model_path /tmp/model.tar.gz
!rm -rf model_neo && mkdir model_neo
!tar -xzvf /tmp/model.tar.gz -C model_neo<jupyter_output><empty_output><jupyter_text>4) Get the game code and have fun!<jupyter_code>import os
import urllib.request
game_url='https://raw.githubusercontent.com/aws-samples/ml-edge-getting-started/main/examples/01_TicTacToe_RL_Tensorflow/tictactoe/game.py'
if not os.path.exists('game.py'):
    urllib.request.urlretrieve(game_url, 'game.py') 
from game import TicTacToeGame
game = TicTacToeGame()
game.run()<jupyter_output><empty_output><jupyter_text>Curious to undertand how the game was implemented?<jupyter_code>!pygmentize game.py<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11997582351163494, 'fraction_numerical': 0.015412511332728921, 'mean_word_length': 5.2218045112781954, 'pattern_counts': {'":': 1, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '659308', 'license': 'MIT-0', 'n_tokens_mistral': 1225, 'n_tokens_neox': 1168, 'n_words': 229, 'path': 'tutorials/02_TicTacToeTF_RL.ipynb', 'repo_name': 'vrakesh/ml-edge-getting-started', 'size': '3309'}