<jupyter_start><jupyter_text>Authenticate with the docker registry first```bashgcloud auth configure-docker```If using TPUs please also authorize Cloud TPU to access your project as [described here](https://cloud.google.com/ml-engine/docs/tensorflow/using-tpusauthorize_your_to_access_your_project).  Set up your output bucket<jupyter_code>BUCKET = "gs://"  # your bucket here
assert re.search(r'gs://.+', BUCKET), 'A GCS bucket is required to store your results.'<jupyter_output><empty_output><jupyter_text>Build a base image to work with fairing<jupyter_code>!cat Dockerfile
!docker build . -t {base_image}
!docker push {base_image}<jupyter_output><empty_output><jupyter_text>Start an AI Platform job<jupyter_code>additional_files = '' # If your code requires additional files, you can specify them here (or include everything in the current folder with glob.glob('./**', recursive=True))
# If your code does not require any dependencies or config changes, you can directly start from an official Tensorflow docker image
#fairing.config.set_builder('docker', registry=DOCKER_REGISTRY, base_image='gcr.io/deeplearning-platform-release/tf-gpu.1-13')

# base image
fairing.config.set_builder('docker', registry=DOCKER_REGISTRY, base_image=base_image)
# AI Platform job hardware config
fairing.config.set_deployer('gcp', job_config={'trainingInput': {'scaleTier': 'CUSTOM', 'masterType': 'standard_p100'}})
# input and output notebooks
fairing.config.set_preprocessor('full_notebook',
                                notebook_file="05K_MNIST_TF20Keras_Tensorboard_playground.ipynb",
                                input_files=additional_files,
                                output_file=os.path.join(BUCKET, 'fairing-output', 'mnist-001.ipynb'))


# GPU settings for single K80, single p100 respectively
# job_config={'trainingInput': {'scaleTier': 'BASIC_GPU'}}
# job_config={'trainingInput': {'scaleTier': 'CUSTOM', 'masterType': 'standard_p100'}}

# These job_config settings for TPUv2
#job_config={'trainingInput': {'scaleTier': 'BASIC_GPU'}}
#job_config={'trainingInput': {'scaleTier': 'CUSTOM', 'masterType': 'n1-standard-8', 'workerType': 'cloud_tpu', 'workerCount': 1,
#                                               'workerConfig': {'accelerator_config': {'type': 'TPU_V2','count': 8}}}})
# On AI Platform, TPUv3 support is alpha and available to whitelisted customers only
fairing.config.run()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12289354706124127, 'fraction_numerical': 0.011508425811755036, 'mean_word_length': 5.1002506265664165, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '444298', 'license': 'Apache-2.0', 'n_tokens_mistral': 783, 'n_tokens_neox': 742, 'n_words': 185, 'path': 'courses/fast-and-lean-data-science/fairing_train.ipynb', 'repo_name': 'Glairly/introduction_to_tensorflow', 'size': '2433'}