<jupyter_start><jupyter_text>TensorFlow- TensorFlow is Googles flagship library for Machine Learning. BackgroundTensorFlow from Google is an open source library that primarily focuses on deep learning. It uses **computational dataflow graphs** to represent models and other calculations.![image.png](attachment:image.png)The nodes in the graph are called **ops (operations)**, whereas the edges denote the multidimensional data tensors transferred between them.TensorFlow is capable of **automatic differentiation**; it stores gradients at each node in the graph, so that no extra computation is used during backpropagation.The library contains many helper functions to create models and execute them on a distributed architecture. The definition of neural networks is generally easier with the **Keras** package. TensorsTensors are collections of real numbers $\mathbb{R}$ with a given dimensionality:| dimensions | common name | example||------------|-------------|--------|| 0 | scalar | single float | | 1 | vector | time series | | 2 | matrix | black/white image | | 3 | 3-tensor | RGB image | | 4 | 4-tensor | RGB video | | 5 | 5-tensor | set of videos | Illustrates how you can perform distributed computing!  What happens if we write a line of python like this?<jupyter_code>3 + 4<jupyter_output><empty_output><jupyter_text>1. Python parses the Python Code(parse tree)2. Python translates the parse tree into elementary operations (byte code)3. Python calls some C++ functions that execute the byte code**Python does this for every single operation!**4. The C++ functions (Python interpreter) exist as machine code that the processor executes directly**Slow, not compiled, but flexible!!**  What TensorFlow does1. Creates a Computation Graph2. Copy the Computation Graph to a device (with TF C library installed). Difference is we can save this graph and take it around. e.g. can save the computation graph to a JSON file3. Each node is linked to code that knows how to execute that node (device-dependent)  Defining a Computation Graph- The easiest operation to use is to define a constant tensor:<jupyter_code>import tensorflow as tf
a = tf.constant(1.0)
b = tf.constant(2.0, tf.float32)
c = a + b
c<jupyter_output><empty_output><jupyter_text>- **Tensor is like a numpy array but with any number of dimensions! Can be 0, 1 or ...**<jupyter_code>type(c)<jupyter_output><empty_output><jupyter_text>- So far we have defined a constant 1.0 and constant 2.0. **These are nodes in the graph**, and this graph is called the **computation graph!****The Computation Graph is the main thing you need to grasp with TensorFlow.**- **We added a 3rd node, the addition!** This makes our computation graph. It isn't doing anything yet as we haven't asked it to.**We can ask it to evaluate the addition by creating an interactive session in Jupyter!**- The sessions can be specific for parts of the computer, here it is run in the local CPU. - Instead, we could say we want it to run it on a GPU (graphics processing unit - devised to make 3D games, can create vectors - became useful for other things e.g. training neural nets, it has a lot of computational power!)- Could also run it on a remote CPU, or a web browser (if you use TF-JS)  Sessions- To execute a computation graph, you need to create a **session** and **evaluate** - using `eval()` one or more nodes. The easiest way to do this is an interactive session (good in Jupyter etc.):<jupyter_code># Here you would specify the local CPU, GPU, remote CPU...
sess = tf.InteractiveSession()

print(c.eval())<jupyter_output>3.0<jupyter_text>- Alternatively, can use `run()`<jupyter_code>sess.run(c)<jupyter_output><empty_output><jupyter_text>- Once we have the computation graph, we can make it run on any of a local CPU, a GPU, a remote CPU, a browser!**TensorFlow decouples from python! It can be exported from our TF program, and sent to an external GPU and run on that** - In a program, you would rather create an explicit session:<jupyter_code>with tf.Session() as sess:
    print(sess.run(c))<jupyter_output>3.0<jupyter_text>Remember, Tensor is like a numpy array but with any number of dimensions! Can be 0, 1, 2 ...<jupyter_code>a = tf.constant([1,2,3], dtype=tf.float32)
a
b = tf.constant(4, dtype=tf.float32)
b
d = tf.random_normal(shape=(3,), dtype=tf.float32)
d
sess = tf.InteractiveSession()
sess.run(d)
c = a + b
e = c * d
c
sess.run(c)
e
sess.run(e)<jupyter_output><empty_output><jupyter_text>- The Python ContextManager `with` closes the session automatically. Variables- **Variables** can change their value inside the graph.<jupyter_code>x = tf.Variable([1.0, 2.0])
y = tf.Variable(tf.random_uniform((2,)))

add = tf.add(x, y)
increase_x = tf.assign(x, x + y)<jupyter_output>WARNING:tensorflow:From /Users/maximcondon/anaconda3/envs/deep_learning/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.<jupyter_text>- All variables need to be initialized with a constant value or another node:<jupyter_code>sess.run(x.initializer)
sess.run(y.initializer)

print(x.eval())
sess.run(add)
print(x.eval())
sess.run(increase_x)
print(x.eval())<jupyter_output>[1.2644954 2.3533587]<jupyter_text>- Alternatively, you can initalize all variables with one op:<jupyter_code>init = tf.global_variables_initializer()

with tf.Session() as sess:
    init.run() # actually initialize all the variables
    result = e.eval()
result<jupyter_output><empty_output><jupyter_text>Placeholders- Placeholders allow you to send data into a computation graph.<jupyter_code>sess = tf.InteractiveSession()
p = tf.placeholder(name='p', shape=(2,), dtype=tf.float32)
q = p * 3

result = sess.run(q, feed_dict={'p:0':[4.0, 5.0]})
print(result)<jupyter_output>[12. 15.]
{'fraction_non_alphanumeric': 0.10253807106598985, 'fraction_numerical': 0.015228426395939087, 'mean_word_length': 5.025484199796127, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '107923', 'license': 'MIT', 'n_tokens_mistral': 1817, 'n_tokens_neox': 1723, 'n_words': 728, 'path': '12_TensorFlow.ipynb', 'repo_name': 'maximcondon/Appendix', 'size': '5910'}