<jupyter_start><jupyter_text>colab specific task*   mount google drive*   change working directory to git repo*   Check TF version*    TPU check<jupyter_code>from google.colab import drive
drive.mount('/content/gdrive')

cd /content/gdrive/My\ Drive/CopyMove/pytfBusterNet/
!pip3 install tensorflow==1.13.1<jupyter_output>Requirement already satisfied: tensorflow==1.13.1 in /usr/local/lib/python3.6/dist-packages (1.13.1)
Requirement already satisfied: tensorboard<1.14.0,>=1.13.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.13.1) (1.13.1)
Requirement already satisfied: tensorflow-estimator<1.14.0rc0,>=1.13.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.13.1) (1.13.0)
Requirement already satisfied: absl-py>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.13.1) (0.8.0)
Requirement already satisfied: numpy>=1.13.3 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.13.1) (1.16.5)
Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.13.1) (1.15.0)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.13.1) (0.33.6)
Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.13.1) (1.12.0)
Requireme[...]<jupyter_text>TPU Check<jupyter_code>import os
import pprint
import tensorflow as tf

if 'COLAB_TPU_ADDR' not in os.environ:
  print('ERROR: Not connected to a TPU runtime; please see the first cell in this notebook for instructions!')
else:
  tpu_address = 'grpc://' + os.environ['COLAB_TPU_ADDR']
  print ('TPU address is', tpu_address)

  with tf.Session(tpu_address) as session:
    devices = session.list_devices()
    
  print('TPU devices:')
  pprint.pprint(devices)

tf.__version__<jupyter_output>TPU address is grpc://10.110.208.90:8470
TPU devices:
[_DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:CPU:0, CPU, -1, 13849900095548455401),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 7087578239193037491),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:0, TPU, 17179869184, 9374415105526213586),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:1, TPU, 17179869184, 12823212531507121852),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:2, TPU, 17179869184, 10834216392966996874),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:3, TPU, 17179869184, 10393830486862736608),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:4, TPU, 17179869184, 16782088074026870289),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:5, TPU, 17179869184, 11914229742950451560),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:6, TPU, 17179869184, 175819143[...]<jupyter_text>Manipulation Model Training  IMPORT<jupyter_code># imports for man_net and loading
from BusterNet.models import man_net
import h5py
import numpy as np

# read data
def readh5(d_path):
    data=h5py.File(d_path, 'r')
    data = np.array(data['data'])
    return data

# load data
d_path=os.path.join(os.getcwd(),'DataSet')
Xp=os.path.join(d_path,'X.h5')
Ymp=os.path.join(d_path,'Ym.h5')

X=readh5(Xp)
Ym=readh5(Ymp)
print(X.shape)
print(Ym.shape)<jupyter_output>(672, 256, 256, 3)
(672, 256, 256, 1)<jupyter_text>Compile Model<jupyter_code>from tensorflow.keras.optimizers import Adam
model=man_net()
model.summary()
model.compile(optimizer=Adam(lr=0.01), loss=tf.keras.losses.binary_crossentropy)<jupyter_output>WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            (None, 256, 256, 3)  0                                            
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 256, 256, 64) 1792        input_1[0][0]                    
__________________________________________________________________________________________________
conv2d_1 (Co[...]<jupyter_text>Convert Model<jupyter_code># This address identifies the TPU we'll use when configuring TensorFlow.
TPU_WORKER = 'grpc://' + os.environ['COLAB_TPU_ADDR']
tf.logging.set_verbosity(tf.logging.INFO)

def convert_model_TPU(model):
  return tf.contrib.tpu.keras_to_tpu_model(model,strategy=tf.contrib.tpu.TPUDistributionStrategy(tf.contrib.cluster_resolver.TPUClusterResolver(TPU_WORKER)))

model=convert_model_TPU(model)<jupyter_output>WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.

INFO:tensorflow:Querying Tensorflow master (grpc://10.110.208.90:8470) for TPU system metadata.
INFO:tensorflow:Found TPU system:
INFO:tensorflow:*** Num TPU Cores: 8
INFO:tensorflow:*** Num TPU Workers: 1
INFO:tensorflow:*** Num TPU Cores Per Worker: 8
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, -1, 13849900095548455401)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 7087578239193037491)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 17179869184, 9374415105526213586)
INFO:tensorflow:*[...]<jupyter_text>Training Parameters<jupyter_code>from sklearn.model_selection import train_test_split
epochs=250
batch_size=30
Xt,Xv,Yt,Yv = train_test_split(X,Ym, test_size=0.2)
print(Xt.shape)
print(Xv.shape)
print(Yt.shape)
print(Yv.shape)<jupyter_output>(537, 256, 256, 3)
(135, 256, 256, 3)
(537, 256, 256, 1)
(135, 256, 256, 1)<jupyter_text>Train<jupyter_code>history=model.fit(Xt,Yt,validation_data=(Xv,Yv),epochs=epochs,batch_size=batch_size, verbose=1)<jupyter_output>Train on 537 samples, validate on 135 samples
Epoch 1/250
INFO:tensorflow:New input shapes; (re-)compiling: mode=train (# of cores 8), [TensorSpec(shape=(3,), dtype=tf.int32, name='core_id0'), TensorSpec(shape=(3, 256, 256, 3), dtype=tf.float32, name='input_1_10'), TensorSpec(shape=(3, 256, 256, 1), dtype=tf.float32, name='conv2d_25_target_30')]
INFO:tensorflow:Overriding default placeholder.
INFO:tensorflow:Cloning Adam {'lr': 0.009999999776482582, 'beta_1': 0.8999999761581421, 'beta_2': 0.9990000128746033, 'decay': 0.0, 'epsilon': 1e-07, 'amsgrad': False}
INFO:tensorflow:Remapping placeholder for input_1
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/contrib/tpu/python/tpu/keras_support.py:302: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
INFO:tensorflow:KerasCrossShard: <tensorflow.python.keras.optimizers.Adam object at 0x7f1d34f6b518> []
WARNING:tensor[...]<jupyter_text>Tarining History<jupyter_code>import matplotlib.pyplot as plt
%matplotlib inline
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model accuracy')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'validation'], loc='upper left')<jupyter_output><empty_output><jupyter_text>Save Model<jupyter_code>model.save_weights('man_net.h5')<jupyter_output>INFO:tensorflow:Copying TPU weights to the CPU
INFO:tensorflow:TPU -> CPU lr: 0.009999999776482582
INFO:tensorflow:TPU -> CPU beta_1: 0.8999999761581421
INFO:tensorflow:TPU -> CPU beta_2: 0.9990000128746033
INFO:tensorflow:TPU -> CPU decay: 0.0
INFO:tensorflow:TPU -> CPU epsilon: 1e-07
WARNING:tensorflow:Cannot update non-variable config: epsilon
INFO:tensorflow:TPU -> CPU amsgrad: False
WARNING:tensorflow:Cannot update non-variable config: amsgrad
{'fraction_non_alphanumeric': 0.1410271758649587, 'fraction_numerical': 0.09840775769184724, 'mean_word_length': 6.340949033391916, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 43, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '58524', 'license': 'MIT', 'n_tokens_mistral': 3450, 'n_tokens_neox': 2992, 'n_words': 550, 'path': 'man.ipynb', 'repo_name': 'mnansary/pytfBusterNet', 'size': '8353'}