<jupyter_start><jupyter_text>Sample for Submarine Experiment SDKThe notebook shows how to use Submarine Experiment SDK to create, get, list, log, delete Submarine Experiment.<jupyter_code>from __future__ import print_function
import submarine
from submarine.experiment.models.environment import Environment
from submarine.experiment.models.experiment_spec import ExperimentSpec
from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
from submarine.experiment.models.experiment_meta import ExperimentMeta<jupyter_output><empty_output><jupyter_text>Create Submarine Client<jupyter_code>submarine_client = submarine.ExperimentClient(host='http://submarine:8080')<jupyter_output><empty_output><jupyter_text>Define TensorFlow experiment spec¶Define Submarine spec¶The demo only creates a worker of TF experiment to run mnist sample.<jupyter_code>experiment_meta = ExperimentMeta(name='mnist',
                                 namespace='submarine',
                                 framework='Tensorflow',
                                 cmd='python /var/tf_mnist/mnist_with_summaries.py'
                                    ' --log_dir=/train/log --learning_rate=0.01'
                                    ' --batch_size=150',
                                 env_vars={'ENV1': 'ENV1'})

worker_spec = ExperimentTaskSpec(resources='cpu=4,memory=2048M',
                                 replicas=1)

environment = Environment(image='gcr.io/kubeflow-ci/tf-mnist-with-summaries:1.0')

experiment_spec = ExperimentSpec(meta=experiment_meta,
                                 environment=environment,
                                 spec={'Worker': worker_spec})<jupyter_output><empty_output><jupyter_text>Create experiment<jupyter_code>experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
experiment<jupyter_output><empty_output><jupyter_text>Get the created experiment<jupyter_code>id = experiment['experimentId']
submarine_client.get_experiment(id)<jupyter_output><empty_output><jupyter_text>List all running experiments<jupyter_code>status = 'running'
submarine_client.list_experiments(status=status)<jupyter_output><empty_output><jupyter_text>Get specific experiment training log<jupyter_code>submarine_client.get_log(id)<jupyter_output>The logs of Pod mnist-worker-0:

WARNING:tensorflow:From /var/tf_mnist/mnist_with_summaries.py:39: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use alternatives such as official/mnist/dataset.py from tensorflow/models.
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Please write your own downloading logic.
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:252: wrapped_fn (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Please use urllib or similar directly.
WARNING:tensorflow:From /us[...]<jupyter_text>Delete the experiment<jupyter_code>submarine_client.delete_experiment(id)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08832159624413145, 'fraction_numerical': 0.009683098591549295, 'mean_word_length': 4.597701149425287, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '464560', 'license': 'Apache-2.0', 'n_tokens_mistral': 1030, 'n_tokens_neox': 987, 'n_words': 181, 'path': 'submarine-sdk/pysubmarine/example/submarine_experiment_sdk.ipynb', 'repo_name': 'cku328/submarine', 'size': '3408'}