<jupyter_start><jupyter_text>What is happening?* I have passed variables in odd fashion.<jupyter_code>sess = tf_reset()

a = tf.placeholder(dtype=tf.float32, shape=[1, 2], name='a_placeholder')
b = tf.placeholder(dtype=tf.float32, shape=[1, 2]) # , name='b_placeholder'

c = a + b

feed1 = np.random.uniform(1, -1, (1, 2))
feed2 = np.random.normal(0, 1, (1, 2))

c1_run = sess.run(c, feed_dict={a: [[1/2, 1]], b: [[9.89, 2.090]]})
c2_run = sess.run(c, feed_dict={a: feed1, b: feed2})
print('c1 = {}'.format(c1_run))
print('c2 = {0}'.format(c2_run))<jupyter_output>c1 = [[10.39  3.09]]
c2 = [[-0.7550663  -0.18605018]]<jupyter_text>What is happening?* tf_reset(): starts tensorflow new session, it is a function defined by me it is not **tensorflow** function.* tf.placeholder: tensorflow func, defines the variable with data type, shape, and name. This info helps when we print it.* 'c = a + b: it tells what operations to do on variables a and b. You can directly write it in the sess.run(.) by replacing c.* sess_run: runs the session and does the operations specified by the user. Values are fed using dictionary. Value **given out by it** is a **numpy array**.* printing the variable 'a', 'b' gives you the variable info.* .format() is just a in-built python print function.<jupyter_code>sess = tf_reset()

a = tf.placeholder(dtype=tf.float32, shape=[None, None], name='a_placeholder')
b = tf.placeholder(dtype=tf.float32, shape=[None, None], name='b_placeholder')

c = a + b

c_run = sess.run(c, feed_dict={a: [[12, 12]], b: [[23, 43]]})
print('a:', a)
print('a shape = {}'.format(a.get_shape()))
print('b:', b)
print('b shape = {}'.format(b.get_shape()))
print('c = {}'.format(c_run))<jupyter_output>a: Tensor("a_placeholder:0", shape=(?, ?), dtype=float32)
a shape = (?, ?)
b: Tensor("b_placeholder:0", shape=(?, ?), dtype=float32)
b shape = (?, ?)
c = [[35. 55.]]<jupyter_text>What is happening?* sess_run: here we see how to run the sess and compute the params. Here we are not performing any operations but just passing and printing it out.* Even though the data type is defined to be float32, but I passed integers.<jupyter_code>sess = tf_reset()

a = tf.constant([[1, 2, 3]], dtype=tf.float32)
b = tf.constant([[1], [2], [3]], dtype=tf.float32)

a_run, b_run = sess.run([a, b])
print('a: \n', a)
print('b: \n', b)
print('a_run: \n', a_run)
print('b_run: \n', b_run)
c = b + b

c_run = sess.run(c) # since a, b are constants you don't feed values to it
print('b:\n{0}'.format(b_run))
print('c:\n{0}'.format(c_run))<jupyter_output>b:
[[1.]
 [2.]
 [3.]]
c:
[[2.]
 [4.]
 [6.]]<jupyter_text>What is happening?* You need to check dimensions of the tensors before feeding. Otherwise you end up getting this. Same issue when using numpy array.<jupyter_code>c = a + b
c_run = sess.run(c)
print('a_run: \n', a_run)
print('b_run: \n', b_run)
print('c_run: \n', c_run)<jupyter_output>a_run: 
 [[1. 2. 3.]]
b_run: 
 [[1.]
 [2.]
 [3.]]
c_run: 
 [[2. 3. 4.]
 [3. 4. 5.]
 [4. 5. 6.]]<jupyter_text>What is happening?* Showing how to do element-wise computation and matrix multiplication.* We can also see how to do multiple computation in one session.<jupyter_code>c_element_wise = a * b
c_matrix = tf.matmul(a, b)

c_run1, c_run2 = sess.run([c_element_wise, c_matrix])
print('Element-wise/Component-wise: \n', c_run1)
print('Matrix multiplication: \n', c_run2)<jupyter_output>Element-wise/Component-wise: 
 [[1. 2. 3.]
 [2. 4. 6.]
 [3. 6. 9.]]
Matrix multiplication: 
 [[14.]]<jupyter_text>What is happening?* Showing how to do chained computations.<jupyter_code>c0 = b + b
c1 = c0 + 1

c0_run, c1_run = sess.run([c0, c1])
print('c0: \n', c0_run)
print('c1: \n', c1_run)<jupyter_output>c0: 
 [[2.]
 [4.]
 [6.]]
c1: 
 [[3.]
 [5.]
 [7.]]<jupyter_text>What is happening?* There are many more in-built functions similar to numpy to compute mean,..., etc.<jupyter_code>c = tf.reduce_mean(b)

c_run = sess.run(c)
print('b_run: \n', b_run)
print('Mean of b_run: \n', c_run)<jupyter_output>b_run: 
 [[1.]
 [2.]
 [3.]]
Mean of b_run: 
 2.0<jupyter_text>What is happening?Now that we can input data and perform computations, we want some of these operations to involve variables that are free parameters, and can be trained using an optimizer (e.g., gradient descent). **Note:** Tensor flow in-built functions have many aliases for the same function. Don't get confused. All of the them will work.Definitions of some important function:* tf.get_variable: creates a tensorflow variable. Gets an existing variable with these parameters or create a new one.* tf.constant_initializer: Initializer that generates tensors with constant values.* tf.global_variables: Returns global variables. Global variables are variables that are shared across machines in a distributed environment. The Variable() constructor or get_variable() automatically adds new variables to the graph collection GraphKeys.GLOBAL_VARIABLES. This convenience function returns the contents of that collection.* tf.global_variables_initializer: initializes global variables in the graph. This is just a shortcut for variables_initializer(global_variables())Flow of the code* First I have created some variables to do computations. Then created a new variable and and check that it's been added to Tensorflow's variables list. We can do operations with the variable just like any other tensor. Before we can run any of these operations, we must first initalize the variables and then we can run the operations just as we normally would.<jupyter_code>sess = tf_reset()

b = tf.constant([[1., 2., 3.]], dtype=tf.float32)

b_run = sess.run(b)
print('b_run: \n', b_run)
var_init_value = [[2.0, 4.0, 6.0]]
var = tf.get_variable(name='myvar', shape=[1, 3], dtype=tf.float32, initializer=tf.constant_initializer(var_init_value))
# you can't rerun this cell because when in same tf session re-creation of the variable 'var' is not allowed.
print(var)
print(tf.global_variables())
c = b + var
print('b: \n', b)
print('var: \n', var)
print('c: \n', c)
init_op = tf.global_variables_initializer()
sess.run(init_op)
c_run = sess.run(c)

print('b_run: \n', b_run)
print('var_init_value: \n', var_init_value)
print('c_run: \n', c_run)<jupyter_output>b_run: 
 [[1. 2. 3.]]
var_init_value: 
 [[2.0, 4.0, 6.0]]
c_run: 
 [[3. 6. 9.]]<jupyter_text>What is happening? How to train a neural network for a simple regression problem?We've discussed how to input data, perform operations, and create variables. We'll now show how to combine all of these **with some minor additions** to train a neural network on a simple regression problem.* First, we'll create data for a 1-dimensional regression problem.* The below code creates the inputs, variables, neural network operations, mean-squared-error loss, gradient descent optimizer, and runs the optimizer using minibatches of the data.* Now that the neural network is trained, we can use it to make predictions.<jupyter_code># generate the data
inputs = np.linspace(-2*np.pi, 2*np.pi, 10000)[:, None]
outputs = np.sin(inputs) + 0.05 * np.random.normal(size=[len(inputs),1])

plt.scatter(inputs[:, 0], outputs[:, 0], s=0.1, color='k', marker='o')
plt.show()
sess = tf_reset()

# NN model
def create_model():
    # Inputs and Outputs
    input_layer = tf.placeholder(dtype=tf.float32, shape=[None, 1])
    output_layer = tf.placeholder(dtype=tf.float32, shape=[None, 1])
    
    # Create Weights
    W0 = tf.get_variable(name='W0', shape=[1, 20], initializer=tf.contrib.layers.xavier_initializer())
    W1 = tf.get_variable(name='W1', shape=[20, 20], initializer=tf.contrib.layers.xavier_initializer())
    W2 = tf.get_variable(name='W2', shape=[20, 1], initializer=tf.contrib.layers.xavier_initializer())
    
    # Bias
    b0 = tf.get_variable(name='b0', shape=[20], initializer=tf.constant_initializer(0.0))
    b1 = tf.get_variable(name='b1', shape=[20], initializer=tf.constant_initializer(0.0))
    b2 = tf.get_variable(name='b2', shape=[1], initializer=tf.constant_initializer(0.0))
    
    # List of all the variables
    weights = [W0, W1, W2]
    bias = [b0, b1, b2]
    activations = [tf.nn.relu, tf.nn.relu, None]
    
    # Forward Pass or create computation graph
    layer = input_layer
    for W, b, phi in zip(weights, bias, activations):
        layer = tf.matmul(layer, W) + b
        if phi is not None:
            layer = phi(layer)
    
    output_pred = layer
    
    return input_layer, output_layer, output_pred

input_layer, output_layer, output_pred = create_model()

# Loss
mse = tf.reduce_mean(0.5 * tf.square(output_pred - output_layer))

# Optimizer
opt = tf.train.AdamOptimizer().minimize(mse)

# Initialize the variables
sess.run(tf.global_variables_initializer())

# Create saver to save model variables
saver = tf.train.Saver()

# Run training
batch_size = 32
for training_step in range(10000):
    # get a random subset of the training data
    indices = np.random.randint(low=0, high=len(inputs), size=batch_size)
    input_batch = inputs[indices]
    output_batch = outputs[indices]
    
    # run the optimizer and get the mse
    _, mse_run = sess.run([opt, mse], feed_dict={input_layer: input_batch, output_layer: output_batch})
    
    # print the mse every so often
    if training_step % 1000 == 0:
        print('{0:04d} mse: {1:.3f}'.format(training_step, mse_run))
        saver.save(sess, '/tmp/model.ckpt')
sess = tf_reset()

# Create the model
input_layer, output_layer, output_pred = create_model()

# Restore the model 
saver = tf.train.Saver()
saver.restore(sess, "/tmp/model.ckpt")

output_pred_run = sess.run(output_pred, feed_dict={input_layer: inputs})

plt.scatter(inputs[:, 0], outputs[:, 0], c='k', marker='o', s=0.1)
plt.scatter(inputs[:, 0], output_pred_run[:, 0], c='r', marker='o', s=0.1)
plt.show()<jupyter_output>INFO:tensorflow:Restoring parameters from /tmp/model.ckpt
{'fraction_non_alphanumeric': 0.14634891536816377, 'fraction_numerical': 0.028108768713718302, 'mean_word_length': 4.595441595441596, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 28, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '393927', 'license': 'MIT', 'n_tokens_mistral': 3599, 'n_tokens_neox': 3423, 'n_words': 1083, 'path': 'TensorFlow v1.13/0-Hello-World/TensorFlow_Practice.ipynb', 'repo_name': 'bvsk35/TensorFlow-V1.13-Examples', 'size': '9819'}