<jupyter_start><jupyter_text>Notes taken while learning Tensorflow  I will use this course as guide:  -  http://web.stanford.edu/class/cs20si/syllabus.html  More info: -  https://github.com/chiphuyen/stanford-tensorflow-tutorials-    Tensoflow Operations+  Constants+  Variables+  Placeholders<jupyter_code># create variables with tf.Variable
s = tf.Variable(2, name="scalar")   
m = tf.Variable([[0, 1], [2, 3]], name="matrix")   
W = tf.Variable(tf.zeros([784,10]))

# create variables with tf.get_variable (this is better)
s = tf.get_variable("scalar", initializer=tf.constant(2))   
m = tf.get_variable("matrix", initializer=tf.constant([[0, 1], [2, 3]]))  
W = tf.get_variable("big_matrix", shape=(784, 10), initializer=tf.zeros_initializer())<jupyter_output><empty_output><jupyter_text>Above we define the initializer but we dont really initialize its value. So we need to call the op when running the session.<jupyter_code>#The easiest way is initializing all variables at once:
with tf.Session() as sess:
	sess.run(tf.global_variables_initializer())

#Initialize only a subset of variables:
with tf.Session() as sess:
	sess.run(tf.variables_initializer([a, b]))

#Initialize a single variable
W = tf.Variable(tf.zeros([784,10]))
with tf.Session() as sess:
	sess.run(W.initializer)<jupyter_output><empty_output><jupyter_text>The eval() op works like running session.run on the variable you pass, it evaluates the value of that node in the graph  print(W.eval()) --------->  Similar to print(sess.run(W)) tf.Variable.assign() op:  -  assigns a value to a variable-  You don’t need to initialize variable because assign_op does it for you. In fact, initializer op is the assign op that assigns the variable’s initial value to the variable itself.<jupyter_code>W = tf.Variable(10)
W.assign(100)
with tf.Session() as sess:
	sess.run(W.initializer)
	print(W.eval()) 				# >> 10

--------

W = tf.Variable(10)
assign_op = W.assign(100)
with tf.Session() as sess:
sess.run(W.initializer)
sess.run(assign_op)
print(W.eval()) 				# >> 100<jupyter_output><empty_output><jupyter_text>Control dependencies in graph:  How to specify order of execution when we have 2 or more independent variables? We use tf.Graph.control_dependencies(control_inputs)<jupyter_code># defines which ops should be run first
# your graph g have 5 ops: a, b, c, d, e
g = tf.get_default_graph()
with g.control_dependencies([a, b, c]):
	# 'd' and 'e' will only run after 'a', 'b', and 'c' have executed.
	d = ...
	e = …<jupyter_output><empty_output><jupyter_text>Placeholders:  -  We use them to supply data to our models, assembling the graph first without knowing the values needed for computation-  tf.placeholder(dtype, shape=None, name=None)How do we supply data to these placeholders?  - We can use a dictionary where the placeholder object is the key and the value is the data we need to pass.<jupyter_code># create a placeholder for a vector of 3 elements, type tf.float32
a = tf.placeholder(tf.float32, shape=[3])

b = tf.constant([5, 5, 5], tf.float32)

# use the placeholder as you would a constant or a variable
c = a + b  # short for tf.add(a, b)

with tf.Session() as sess:
	print(sess.run(c, feed_dict={a: [1, 2, 3]})) 	# the tensor a is the key, not the string ‘a’

# >> [6, 7, 8]<jupyter_output><empty_output><jupyter_text>Important note: shape=None means that tensor of any shape will be accepted as value for placeholder. shape=None is easy to construct graphs, but nightmarish for debugging.  shape=None also breaks all following shape inference, which makes many ops not work because they expect certain rank. If we want to feed multiple data points, we need to do it one at a time.<jupyter_code>with tf.Session() as sess:
	for a_value in list_of_values_for_a:
	print(sess.run(c, {a: a_value}))
    
# How to know when can I feed data into a tensor, is this tensor a ph?
tf.Graph.is_feedable(tensor) 
# True if and only if tensor is feedable.<jupyter_output><empty_output><jupyter_text>Extremely helpful for testing: Feed in dummy values to test parts of a large graph<jupyter_code>#We can feed values to ops such as add 
# create operations, tensors, etc (using the default graph)
a = tf.add(2, 5)
b = tf.multiply(a, 3)

with tf.Session() as sess:
	# compute the value of b given a is 15
	sess.run(b, feed_dict={a: 15}) 				# >> 45<jupyter_output><empty_output><jupyter_text>Lazy Loading (Defer creating/initializing an object until it is needed)<jupyter_code>#Normal loading
x = tf.Variable(10, name='x')
y = tf.Variable(20, name='y')
z = tf.add(x, y) 		# create the node before executing the graph

writer = tf.summary.FileWriter('./graphs/normal_loading', tf.get_default_graph())
with tf.Session() as sess:
	sess.run(tf.global_variables_initializer())
	for _ in range(10):
		sess.run(z)
writer.close()
#############################
#Lazy loading
x = tf.Variable(10, name='x')
y = tf.Variable(20, name='y')

writer = tf.summary.FileWriter('./graphs/normal_loading', tf.get_default_graph())
with tf.Session() as sess:
	sess.run(tf.global_variables_initializer())
	for _ in range(10):
		sess.run(tf.add(x, y)) # someone decides to be clever to save one line of code
writer.close()<jupyter_output><empty_output><jupyter_text>Both give the same results, but whats the problem?  let's execute tf.get_default_graph().as_graph_def()<jupyter_code>#Normal loading
node {
  name: "Add"
  op: "Add"
  input: "x/read"
  input: "y/read"             #Node “Add” added once to the graph definition
  attr {
    key: "T"
    value {
      type: DT_INT32
    }
  }
}
#############################
#Lazy loading                  Node “Add” added 10 times to the graph definition, Or as many times as you want to compute z
node {
  name: "Add_1"
  op: "Add"
  ...
  }
...
node {
  name: "Add_10"
  op: "Add"
  ...
}<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12920962199312716, 'fraction_numerical': 0.015292096219931271, 'mean_word_length': 4.554389312977099, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 46, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '245938', 'license': 'MIT', 'n_tokens_mistral': 2039, 'n_tokens_neox': 1937, 'n_words': 657, 'path': 'TensorFlow Notes.ipynb', 'repo_name': 'WencesVillegasMarset/stanford-tensorflow-tutorials', 'size': '5820'}