<jupyter_start><jupyter_text>Running TF Profiler (w/ Keras)  Prerequisites- Python 3.6- Keras <= 2.1.6- Compatible Tensorflow- Profiler GUI prerequisites: https://github.com/tensorflow/profiler-ui. It's important that you install the right prerequisites Keras doesn't currently support runtime options: https://github.com/tensorflow/tensorflow/issues/19911The rest of the necessary libraries (and Python) come preinstalled with Anaconda and it's recommended that you setup the prerequisites in a virtual environment so it doesn't interfere with other projects.<jupyter_code>import keras
import tensorflow as tf<jupyter_output>C:\Users\vsunkavalli\AppData\Local\conda\conda\envs\profilerenv\lib\site-packages\h5py\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.<jupyter_text>Creating your modelMake sure that you compile your models with tensorflow runtime options like shown:    run_metadata = tf.RunMetadata()    run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)    model.compile(..., run_metadata=run_metadata, run_options=run_options)<jupyter_code>#Loading Data - Unimportant
from keras.datasets import cifar10
num_classes=10

(x_train, y_train), (x_test, y_test) = cifar10.load_data()

x_train = x_train[0:512]
y_train = y_train[0:512]
x_test = x_test[0:64]
y_test = y_test[0:64]

print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# Convert class vectors to binary class matrices.
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255
#Loading Model - Unimportant
from keras.datasets import cifar10
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Conv2D, MaxPooling2D

batch_size = 32
epochs = 3


model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same',
                 input_shape=x_train.shape[1:]))
model.add(Activation('relu'))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes))
model.add(Activation('softmax'))

opt = keras.optimizers.rmsprop(lr=0.0001, decay=1e-6)
run_metadata = tf.RunMetadata()
run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'],
              run_metadata=run_metadata, options=run_options)<jupyter_output><empty_output><jupyter_text>Profiling your modelWhen training your model, you must setup the code in a particular way in order to get the profile log. Make sure to follow the code provided and only edit the train statement.<jupyter_code>log_dir = './' #Set up the directory where you want to store your profiles
#Using TF Profiler

builder = tf.profiler.ProfileOptionBuilder 
#ProfileOptionBuilder is an object that lets us create options to profile(bytes used, cpu time, etc.)
opts = builder(builder.time_and_memory()).order_by('micros').build()
#The first option is custom: reading execution times and memory usage and 
#sorting it by the time it took to run('micros' means microseconds)
opts2 = tf.profiler.ProfileOptionBuilder.trainable_variables_parameter()
#The second option is prebuilt: options used to profile the 
#parameters used for trainable variables(layer size, depth. stride, etc.)

#The bottom part instantiates when the profiler runs and where the log saves
with tf.contrib.tfprof.ProfileContext(
    log_dir,
    trace_steps=range(10,20),
    dump_steps=[20]) as pctx:

    #These functions automatically profile the two options we 
    #created under names and run them during the steps specified 
    pctx.add_auto_profiling('op', opts, [15,18,20])
    pctx.add_auto_profiling('scope', opts2, [20])
    model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_data=(x_test, y_test),shuffle=True)<jupyter_output>Train on 512 samples, validate on 64 samples
Epoch 1/3
256/512 [==============>...............] - ETA: 2s - loss: 2.3187 - acc: 0.1016
{'fraction_non_alphanumeric': 0.11704618003830602, 'fraction_numerical': 0.02958076186422643, 'mean_word_length': 5.791907514450867, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '277707', 'license': 'MIT', 'n_tokens_mistral': 1635, 'n_tokens_neox': 1538, 'n_words': 418, 'path': 'Profiler Demo.ipynb', 'repo_name': 'vinsunkavalli/tfprofiler_demo', 'size': '4699'}