<jupyter_start><jupyter_text>Copyright (c) Microsoft Corporation.  Licensed under the MIT License. FWI in Azure project Create Experimentation Docker imageFWI demo based on: This project ports devito (https://github.com/opesci/devito) into Azure and runs tutorial notebooks at:https://nbviewer.jupyter.org/github/opesci/devito/blob/master/examples/seismic/tutorials/In this notebook we create a custom docker image that will be used to run the devito demo notebooks in AzureML.  - We transparently create a docker file, a conda environment .yml file, build the docker image and push it into dockerhub. Azure ACR could also be used for storing docker images.  - The conda environment .yml file lists conda and pip installs, and separates all python dependencies from the docker installs.  - The dockerfile is generic. The only AzureML depedency is azureml-sdk pip installable package in conda environment .yml file - The created docer image will be run in following notebook in a container on the local AzureVM or on a remote AzureML compute cluster. This AzureML pattern decouples experimentation (or training) job definition (experimentation script, data location, dependencies and docker image) happening on the control plane machine that runs this notebook, from the elastically allocated and Azure managed VM/cluster that does the actual training/experimentation computation. User input requiring steps: - [Fill in and save docker image name settings, if needed. ](docker_image_settings) - [Update DOCKER_CONTAINER_MOUNT_POINT to match our local path](docker_image_settings) - [Set docker build and test flags](docker_build_test_settings)<jupyter_code># Allow multiple displays per cell
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all" 
import sys, os
import shutil
import urllib

import platform
import math
import docker
platform.platform()
os.getcwd()<jupyter_output><empty_output><jupyter_text>Setup docker image build and test process.  - devito tests take abou 15 mins (981.41 seconds). When running this notebook for first time make:     > docker_build_no_cache = '--no-cache'       > docker_test_run_devito_tests = True     [Back](user_input_requiring_steps) to summary of user input requiring steps.<jupyter_code>docker_build_no_cache = ''  # '--no-cache' # or '' #
docker_test_run_devito_tests = True # True # False<jupyter_output><empty_output><jupyter_text>Import utilities functions<jupyter_code>def add_path_to_sys_path(path_to_append):
    if not (any(path_to_append in paths for paths in sys.path)):
        sys.path.append(path_to_append)
        
auxiliary_files_dir = os.path.join(*(['.', 'src']))
paths_to_append = [os.path.join(os.getcwd(), auxiliary_files_dir)]
[add_path_to_sys_path(crt_path) for crt_path in paths_to_append]

import project_utils
prj_consts = project_utils.project_consts()<jupyter_output><empty_output><jupyter_text>Create experimentation docker file<jupyter_code>dotenv_file_path = os.path.join(*(prj_consts.DOTENV_FILE_PATH))
dotenv_file_path
!pwd
# azureml_sdk_version set here must match azureml sdk version pinned in conda env file written to conda_common_file_path below
azureml_sdk_version = '1.0.79'<jupyter_output><empty_output><jupyter_text>Input here docker image settings in cell below we use [dotenv](https://github.com/theskumar/python-dotenv) to overwrite docker image properties already save in dotenv_file_path. Change as needed, e.g. update azureml_sdk version if using a different version.[Back](user_input_requiring_steps) to summary of user input requiring steps.<jupyter_code># SDK changes often, so we'll keep its version transparent 
import dotenv

# EXPERIMENTATION_IMAGE_VERSION should:
# - match sdk version in fwi01_conda_env01 environmnet in conda_env_fwi01_azureml_sdk.v1.0.XX.yml file below
# -  match the conda env yml file name, e.g. conda_env_fwi01_azureml_sdk.v1.0.xx.yml referenced in 
#      Dockerfile_fwi01_azureml_sdk.v1.0.xx
# dotenv.set_key(dotenv_file_path, 'EXPERIMENTATION_DOCKER_IMAGE_NAME', 'fwi01_azureml')
dotenv.set_key(dotenv_file_path, 'EXPERIMENTATION_DOCKER_IMAGE_TAG', ('sdk.v'+azureml_sdk_version))


docker_container_mount_point = os.getcwd()
# or something like "/datadrive01/prj/DeepSeismic/contrib/fwi/azureml_devito/notebooks'
dotenv.set_key(dotenv_file_path, 'DOCKER_CONTAINER_MOUNT_POINT', docker_container_mount_point)
%reload_ext dotenv
%dotenv $dotenv_file_path

docker_file_location = os.path.join(*(prj_consts.AML_EXPERIMENT_DIR + ['docker_build']))

docker_file_name = 'Dockerfile'+ '_' + os.getenv('EXPERIMENTATION_DOCKER_IMAGE_NAME')

conda_dependency_file_name = 'conda_env'+ '_' + os.getenv('EXPERIMENTATION_DOCKER_IMAGE_NAME')
conda_dependency_common_file_name = conda_dependency_file_name

devito_conda_dependency_file_name = 'devito_conda_env'+'.yml'

docker_repo_name = os.getenv('ACR_NAME')+'.azurecr.io' # or os.getenv('DOCKER_LOGIN')
docker_image_name = docker_repo_name + '/' + os.getenv('EXPERIMENTATION_DOCKER_IMAGE_NAME')

image_version = os.getenv('EXPERIMENTATION_DOCKER_IMAGE_TAG')
if image_version!="":
    docker_file_name = docker_file_name +'_'+ image_version
    conda_dependency_file_name = conda_dependency_file_name+'_'+ image_version
    docker_image_name = docker_image_name +':'+ image_version
conda_dependency_file_name=conda_dependency_file_name+'.yml'
conda_dependency_common_file_name = conda_dependency_common_file_name+'.yml'

docker_file_dir = os.path.join(*([os.getcwd(), docker_file_location]))
os.makedirs(docker_file_dir, exist_ok=True)
docker_file_path = os.path.join(*([docker_file_dir]+[docker_file_name]))
conda_file_path = os.path.join(*([docker_file_dir]+[conda_dependency_file_name]))
conda_common_file_path = os.path.join(*([docker_file_dir]+[conda_dependency_common_file_name]))

docker_image_name

conda_dependency_file_name
conda_file_path
conda_common_file_path

docker_file_dir
docker_file_path
%%writefile $conda_common_file_path
name: fwi01_conda_env01
    
#https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.13.1-cp37-cp37m-linux_x86_64.whl    
# https://github.com/dask/dask-tutorial

channels:
  - anaconda
  - conda-forge
dependencies:
  - python=3.6 # 3.6 req by tf, not 3.7.2 
  - dask
  - distributed
  - h5py
  - matplotlib
  - nb_conda
  - notebook 
  - numpy 
  - pandas
  - pip
  - py-cpuinfo # all required by devito or dask-tutorial
  - pytables
  - python-graphviz
  - requests
  - pillow
  - scipy
  - snakeviz
  - scikit-image
  - toolz
  - pip:
    - anytree # required by devito
    - azureml-sdk[notebooks,automl]==1.0.79
    - codepy # required by devito
    - papermill[azure]
    - pyrevolve # required by devito
%%writefile $docker_file_path 

FROM continuumio/miniconda3:4.7.12    
MAINTAINER <NAME> <<EMAIL>>

RUN apt-get update --fix-missing && apt-get install -y --no-install-recommends \
    gcc g++ \
    wget bzip2 \
    curl \
    git make \
    mpich \ 
    libmpich-dev && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

ENV CONDA_ENV_FILE_NAME conda_env_fwi01_azureml.yml
ADD $CONDA_ENV_FILE_NAME /tmp/$CONDA_ENV_FILE_NAME
ENV CONDA_DIR /opt/conda
ENV CONDA_ENV_NAME fwi01_conda_env

RUN git clone https://github.com/opesci/devito.git  && \
    cd devito  && \
    /opt/conda/bin/conda env create -q --name $CONDA_ENV_NAME -f environment.yml && \
    pip install -e . 
    
ENV CONDA_AUTO_UPDATE_CONDA=false
ENV CONDA_DEFAULT_ENV=$CONDA_ENV_NAME
ENV CONDA_PREFIX=$CONDA_DIR/envs/$CONDA_DEFAULT_ENV
ENV PATH=$CONDA_PREFIX/bin:/opt/conda/bin:$PATH   

RUN /opt/conda/bin/conda env update --name $CONDA_ENV_NAME -f /tmp/$CONDA_ENV_FILE_NAME && \
    /opt/conda/bin/conda clean  --yes --all

ENV PYTHONPATH=$PYTHONPATH:devito/app

# WORKDIR /devito     
    
CMD /bin/bash
shutil.copyfile(conda_common_file_path, conda_file_path)

! ls -l $docker_file_dir
cli_command='docker build -t '+ docker_image_name + \
' -f ' + docker_file_path + \
' ' + docker_file_dir + ' ' +\
docker_build_no_cache  #'' #' --no-cache'


cli_command
docker_build_response = ! $cli_command

docker_build_response[0:5] 
docker_build_response[-5:]<jupyter_output><empty_output><jupyter_text>Docker containers can be run using python docker sdk<jupyter_code>docker_image_name

sh_command='bash -c "pwd;python -c \'import azureml.core;print(azureml.core.VERSION)\'"'
sh_command
client = docker.from_env()
client.containers.run(docker_image_name, 
                      remove=True,
                      volumes={os.getenv('DOCKER_CONTAINER_MOUNT_POINT'): {'bind': '/workspace', 'mode': 'rw'}},
                      working_dir='/',
                      command=sh_command)<jupyter_output><empty_output><jupyter_text>Docker containers can also be run in cli Here we also create a log file to capture commands execution in container. If flag docker_test_run_devito_tests is True, we run and capture test commands output. Tests take abou 15 minutes to run. If flag docker_test_run_devito_tests is False, we show the results of a previous session.<jupyter_code>fwi01_log_file = os.path.join(*(['.', 'fwi01_azureml_buildexperimentationdockerimage.log']))
fwi01_log_file<jupyter_output><empty_output><jupyter_text>Create command for running devito tests, capture output in a log file, save log file outside container<jupyter_code>if docker_test_run_devito_tests:
    run_devito_tests_command = ' python -m pytest tests/ '   + \
'> ' + fwi01_log_file +' 2>&1; ' + \
' mv ' + fwi01_log_file + ' /workspace/'  
    
    with open(os.path.join(*(['.', 'fwi01_azureml_buildexperimentationdockerimage.log'])), "w") as crt_log_file:
        print('Before running e13n container... ', file=crt_log_file)
    print('\ncontent of devito tests log file before testing:')
    !cat $fwi01_log_file
else:
    run_devito_tests_command =  '' 

# run_devito_tests_command =  'ls -l > ./fwi01_azureml_buildexperimentationdockerimage.log 2>&1;  mv ./fwi01_azureml_buildexperimentationdockerimage.log /workspace/'
run_devito_tests_command
cli_command='docker run -it --rm  --name fwi01_azureml_container ' +\
' -v '+os.getenv('DOCKER_CONTAINER_MOUNT_POINT')+':/workspace:rw ' + \
docker_image_name + \
' /bin/bash -c "conda env list ; ls -l /devito/tests;  '  + \
'python -c \'import azureml.core;print(azureml.core.VERSION)\'; '  + \
'cd /devito; '  + \
run_devito_tests_command +\
' "'

cli_command
! $cli_command
# # ============= 774 passed, 70 skipped, 1 xfailed in 1106.76 seconds =============
print('\ncontent of devito tests log file after testing:')
!cat $fwi01_log_file<jupyter_output><empty_output><jupyter_text>Use the ACR created in previous notebook or docker hub to push your image<jupyter_code># docker_pwd = os.getenv('DOCKER_PWD')
# docker_login = os.getenv('DOCKER_LOGIN')
# !docker login -u=$docker_login -p=$docker_pwd
# !docker push {docker_image_name}

%dotenv -o $dotenv_file_path
cli_command='az acr login --name '+os.getenv('ACR_NAME')
# print cli command
cli_command

# run cli command
cli_command = cli_command+' --username '+os.getenv('ACR_USERNAME') + ' --password ' + os.getenv('ACR_PASSWORD')
! $cli_command
cli_command='docker push '+docker_image_name
cli_command
! $cli_command
# !jupyter nbconvert 010_CreateExperimentationDockerImage_GeophysicsTutorial_FWI_Azure_devito --to html
print('Finished running 010_CreateExperimentationDockerImage_GeophysicsTutorial_FWI_Azure_devito!')<jupyter_output>Finished running 010_CreateExperimentationDockerImage_GeophysicsTutorial_FWI_Azure_devito!
{'fraction_non_alphanumeric': 0.10019200558561704, 'fraction_numerical': 0.010734857741316111, 'mean_word_length': 5.204114780725501, 'pattern_counts': {'":': 1, '<': 39, '<?xml version=': 0, '>': 45, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '425344', 'license': 'MIT', 'n_tokens_mistral': 4177, 'n_tokens_neox': 3958, 'n_words': 881, 'path': 'contrib/fwi/azureml_devito/notebooks/010_CreateExperimentationDockerImage_GeophysicsTutorial_FWI_Azure_devito.ipynb', 'repo_name': 'tobi-ore/seismic-deeplearning', 'size': '11484'}