<jupyter_start><jupyter_text>Tensorflow-2.0 in Action  > TensorFlow is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices.> Originally developed by researchers and engineers from the Google Brain team within Google’s AI organization, it comes with strong support for machine learning and deep learning and the flexible numerical computation core is used across many other scientific domains.  Installation of Tensorflow >TensorFlow is tested and supported on the following 64-bit systems:>1.Ubuntu 16.04 or later>2.Windows 7 or later>3.macOS 10.12.6 (Sierra) or later (no GPU support)>4.Raspbian 9.0 or later > **pip install tensorflow==2.0.0**> To run from Anaconda Prompt> **!pip install tensorflow==2.0.0**> To run from Jupyter Notebook `Both Tensorflow 2.0 and Keras have been released for four years (Keras was released in March 2015, and Tensorflow was released in November of the same year). The rapid development of deep learning in the past days, we also know some problems of Tensorflow1.x and Keras:`* Using Tensorflow means programming static graphs, which is difficult and inconvenient for programs that are familiar with imperative programming* Tensorflow api is powerful and flexible, but it is more complex, confusing and difficult to use.* Keras api is productive and easy to use, but lacks flexibility for research  Version Check<jupyter_code>import tensorflow as tf
print("TensorFlow version: {}".format(tf.__version__))
print("Keras version: {}".format(tf.keras.__version__))

import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns<jupyter_output>TensorFlow version: 2.4.1
Keras version: 2.4.0<jupyter_text>`Tensorflow2.0 is a combination design of Tensorflow1.x and Keras. Considering user feedback and framework development over the past four years, it largely solves the above problems and will become the future machine learning platform.`> Tensorflow 2.0 is built on the following core ideas:* The coding is more pythonic, so that users can get the results immediately like they are programming in numpy* Retaining the characteristics of static graphs (for performance, distributed, and production deployment), this makes TensorFlow fast, scalable, and ready for production.* Using Keras as a high-level API for deep learning, making Tensorflow easy to use and efficient* Make the entire framework both high-level features (easy to use, efficient, and not flexible) and low-level features (powerful and scalable, not easy to use, but very flexible) >Eager execution is by default in TensorFlow 2.0 and, it needs no special setup.>The following below code can be used to find out whether a CPU or GPU is in use  GPU/CPU Check<jupyter_code>variable = tf.Variable([3, 3])
if tf.test.is_gpu_available():
    print('GPU')
    print('GPU #0?')
    print(var.device.endswith('GPU:0'))
else:
    print('CPU')<jupyter_output>CPU<jupyter_text>Tensor Constant<jupyter_code>ineuron = tf.constant(42)
ineuron
ineuron.numpy()
ineuron1 = tf.constant(1, dtype = tf.int64)
ineuron1
ineuron_x = tf.constant([[4,2],[9,5]])
print(ineuron_x)
ineuron_x.numpy()
print('shape:',ineuron_x.shape)
print(ineuron_x.dtype)<jupyter_output>shape: (2, 2)
<dtype: 'int32'><jupyter_text>Commonly used method is to generate constant tf.ones and the tf.zeros like of numpy np.ones & np.zeros<jupyter_code>print(tf.ones(shape=(2,3)))
print(tf.zeros(shape=(3,2)))
const2 = tf.constant([[3,4,5], [3,4,5]]);tf
const1 = tf.constant([[1,2,3], [1,2,3]]);
result = tf.add(const1, const2);
print(result)<jupyter_output>Tensor("Add:0", shape=(2, 3), dtype=int32)<jupyter_text>>We have defined two constants and we add one value to the other. >As a result, we got a Tensor object with the result of the adding.   Random constant<jupyter_code>tf.random.normal(shape=(2,2),mean=0,stddev=1.0)
tf.random.uniform(shape=(2,2),minval=0,maxval=10,dtype=tf.int32)<jupyter_output><empty_output><jupyter_text>Variables >A variable is a special tensor that is used to store variable values ​​and needs to be initialized with some values  Declaring variables<jupyter_code>var0 = 24 # python variable
var1 = tf.Variable(42) # rank 0 tensor
var2 = tf.Variable([ [ [0., 1., 2.], [3., 4., 5.] ], [ [6., 7., 8.], [9., 10., 11.] ] ]) #rank 3 tensor
var0, var1, var2<jupyter_output><empty_output><jupyter_text>>TensorFlow will infer the datatype, defaulting to tf.float32 for floats and tf.int32 for integers  The datatype can be explicitly specified<jupyter_code>float_var64 = tf.Variable(89, dtype = tf.float64)
float_var64.dtype<jupyter_output><empty_output><jupyter_text>`TensorFlow has a large number of built-in datatypes.`* tf.float16: 16-bit half-precision floating-point.* tf.float32: 32-bit single-precision floating-point.* tf.float64: 64-bit double-precision floating-point.* tf.bfloat16: 16-bit truncated floating-point.* tf.complex64: 64-bit single-precision complex.* tf.complex128: 128-bit double-precision complex.* tf.int8: 8-bit signed integer.* tf.uint8: 8-bit unsigned integer.* tf.uint16: 16-bit unsigned integer.* tf.uint32: 32-bit unsigned integer.* tf.uint64: 64-bit unsigned integer.* tf.int16: 16-bit signed integer.* tf.int32: 32-bit signed integer.* tf.int64: 64-bit signed integer.* tf.bool: Boolean.* tf.string: String.* tf.qint8: Quantized 8-bit signed integer.* tf.quint8: Quantized 8-bit unsigned integer.* tf.qint16: Quantized 16-bit signed integer.* tf.quint16: Quantized 16-bit unsigned integer.* tf.qint32: Quantized 32-bit signed integer.* tf.resource: Handle to a mutable resource.* tf.variant: Values of arbitrary types.  To reassign a variable, use var.assign()<jupyter_code>var_reassign = tf.Variable(89.)
var_reassign
var_reassign.assign(98.)
var_reassign
initial_value = tf.random.normal(shape=(2,2))
a = tf.Variable(initial_value)
print(a)<jupyter_output><tf.Variable 'Variable_5:0' shape=(2, 2) dtype=float32_ref><jupyter_text>>We can assign "=" with assign (value), or assign_add (value) with "+ =", or assign_sub (value) with "-="<jupyter_code>new_value = tf.random.normal(shape=(2, 2))
# a.assign(new_value)
# print(a.numpy())
for i in range(2):
    for j in range(2):
        assert a[i, j] == new_value[i, j] # This thing dosen't work. I don't know why Ineuron has put this in their training notebook.
new_value,a
added_value = tf.random.normal(shape=(2,2))
a.assign_add(added_value)
for i in range(2):
    for j in range(2):
        assert a[i,j] == new_value[i,j]+added_value[i,j]<jupyter_output><empty_output><jupyter_text>Shaping a tensor<jupyter_code>tensor = tf.Variable([ [ [10., 11., 12.], [13., 14., 15.] ], [ [16., 17., 18.], [19., 20., 21.] ] ]) # tensor variable
print(tensor.shape)<jupyter_output>(2, 2, 3)<jupyter_text>Tensors can be reshaped and retain the same values which is required for constructing Neural networks.<jupyter_code>tensor1 = tf.reshape(tensor,[2,6]) # 2 rows 6 cols
#tensor2 = tf.reshape(tensor,[1,12]) # 1 rows 12 cols
tensor1
tensor2 = tf.reshape(tensor,[1,12]) # 1 row 12 columns
tensor2<jupyter_output><empty_output><jupyter_text>Rank of a tensor >The rank of a tensor is defined as the number of dimensions, which is the number of indices that are required to specify any particular element of that tensor.<jupyter_code>tf.rank(tensor)<jupyter_output><empty_output><jupyter_text>>(the shape is () because the output here is a scalar value)  Specifying an element of a tensor<jupyter_code>tensor3 = tensor[1, 0, 2] # slice 1, row 0, column 2
tensor3<jupyter_output><empty_output><jupyter_text>Casting a tensor to a NumPy variable<jupyter_code>print(tensor.numpy())
print(tensor[1, 0, 2].numpy())<jupyter_output>18.0<jupyter_text>Finding the size or length of a tensor<jupyter_code>tensor_size = tf.size(input=tensor).numpy()
tensor_size
#the datatype of a tensor
tensor3.dtype<jupyter_output><empty_output><jupyter_text>Tensorflow mathematical operations>Can be used as numpy for artificial operations. Tensorflow can not execute these operations on the GPU or TPU.<jupyter_code>a = tf.random.normal(shape=(2,2))
b = tf.random.normal(shape=(2,2))
c = a + b
d = tf.square(c)
e = tf.exp(c)
print(a)
print(b)
print(c)
print(d)
print(e)
aa,ab = tf.Variable(2),tf.Variable(7)
aa**ab<jupyter_output><empty_output><jupyter_text>Performing element-wise primitive tensor operations<jupyter_code>tensor*tensor<jupyter_output><empty_output><jupyter_text>Broadcasting in Tensorflow>Element-wise tensor operations support broadcasting in the same way that NumPy arrays do.>The simplest example is multiplication of  a tensor by a scalar value.<jupyter_code>tensor4 = tensor*4
print(tensor4)<jupyter_output>tf.Tensor(
[[[40. 44. 48.]
  [52. 56. 60.]]

 [[64. 68. 72.]
  [76. 80. 84.]]], shape=(2, 2, 3), dtype=float32)<jupyter_text>Transpose Matrix multiplication<jupyter_code>matrix_u = tf.constant([[6,7,6]])
matrix_v = tf.constant([[3,4,3]])
tf.matmul(matrix_u, tf.transpose(a=matrix_v))<jupyter_output><empty_output><jupyter_text>Casting a tensor to another datatype<jupyter_code>i = tf.cast(tensor1, dtype=tf.int32)
i
t = tf.constant(tensor1,dtype=tf.int32)
t
t = tf.constant(tensor1.numpy(),dtype=tf.int32)
t<jupyter_output><empty_output><jupyter_text>Casting with truncation<jupyter_code>j = tf.cast(tf.constant(4.9), dtype=tf.int32)
j<jupyter_output><empty_output><jupyter_text>Ragged tensors `A ragged tensor is a tensor having one or more ragged dimensions. Ragged dimensions are dimensions that have slices having various lengths.There are a variety of methods for the declaration of ragged arrays, the simplest way is declaring a constant ragged array.`  Below example shows how to declare a constant ragged array<jupyter_code>ragged =tf.ragged.constant([[9, 7, 4, 3], [], [11, 12, 8], [3], [7,8]])
print(ragged)
print(ragged[0,:])
print(ragged[1,:])
print(ragged[2,:])
print(ragged[3,:])
print(ragged[4,:])<jupyter_output><tf.RaggedTensor [[9, 7, 4, 3], [], [11, 12, 8], [3], [7, 8]]>
tf.Tensor([9 7 4 3], shape=(4,), dtype=int32)
tf.Tensor([], shape=(0,), dtype=int32)
tf.Tensor([11 12  8], shape=(3,), dtype=int32)
tf.Tensor([3], shape=(1,), dtype=int32)
tf.Tensor([7 8], shape=(2,), dtype=int32)<jupyter_text>Squared difference of tensors<jupyter_code>varx = [4,5,6,1,2]
varxx = [4,15,6,1,-3]
vary = 8
varz = tf.math.squared_difference(varx,varxx)
varzz = tf.math.squared_difference(varx,vary) 
varz,varzz<jupyter_output><empty_output><jupyter_text>Calculate the mean >Function available>tf.reduce_mean(). `Similar to np.mean, except that it infers the return datatype from the input tensor, whereas np.mean allows you to specify the output type``tf.reduce_mean(input_tensor, axis=None, keepdims=None, name=None)`<jupyter_code># Defining a constant
numbers = tf.constant([[8., 9.], [1., 2.]])<jupyter_output><empty_output><jupyter_text>Calculate the mean across all axes<jupyter_code>tf.reduce_mean(input_tensor=numbers) #default axis = None<jupyter_output><empty_output><jupyter_text>Calculate the mean across columns (reduce rows) with this:<jupyter_code>tf.reduce_mean(input_tensor=numbers, axis=0)<jupyter_output><empty_output><jupyter_text>When keepdims = True<jupyter_code>tf.reduce_mean(input_tensor=numbers, axis=0, keepdims=True) #the reduced axis is retained with a length of 1<jupyter_output><empty_output><jupyter_text>Calculate the mean across rows (reduce columns) with this:<jupyter_code>tf.reduce_mean(input_tensor=numbers, axis=1)<jupyter_output><empty_output><jupyter_text>When keepdims= True<jupyter_code>tf.reduce_mean(input_tensor=numbers, axis=1, keepdims=True) #the reduced axis is retained with a length of 1<jupyter_output><empty_output><jupyter_text>Random values generation  tf.random.normal()>tf.random.normal() outputs a tensor of the given shape filled with values of the dtype type from a normal distribution.>The function is as follows:    `tf.random.normal(shape, mean = 0, stddev =2, dtype=tf.float32, seed=None, name=None)`<jupyter_code>tf.random.normal(shape = (3,2), mean=10, stddev=2, dtype=tf.float32, seed=None, name=None)
randon_num = tf.random.normal(shape = (3,2), mean=10.0, stddev=2.0)
print(randon_num)<jupyter_output>tf.Tensor(
[[13.419874  9.652067]
 [ 8.206388 10.20897 ]
 [ 9.400301  9.270695]], shape=(3, 2), dtype=float32)<jupyter_text>tf.random.uniform() >The function is this:    >tf.random.uniform(shape, minval = 0, maxval= None, dtype=tf.float32, seed=None, name=None) `This outputs a tensor of the given shape filled with values from a uniform distribution in the range minval to maxval, where the lower bound is inclusive but the upper bound isn't.Example:`<jupyter_code>tf.random.uniform(shape = (2,4), minval=0, maxval=None, dtype=tf.float32, seed=None, name=None)<jupyter_output><empty_output><jupyter_text>Setting the seed<jupyter_code>tf.random.set_seed(11)
random_num1 = tf.random.uniform(shape = (2,2), maxval=10, dtype = tf.int32)
random_num2 = tf.random.uniform(shape = (2,2), maxval=10, dtype = tf.int32)
print(random_num1) #Call 1
print(random_num2)
tf.random.set_seed(11) #same seed
random_num1 = tf.random.uniform(shape = (2,2), maxval=10, dtype = tf.int32)
random_num2 = tf.random.uniform(shape = (2,2), maxval=10, dtype = tf.int32)
print(random_num1) #Call 2
print(random_num2)<jupyter_output>tf.Tensor(
[[4 6]
 [5 2]], shape=(2, 2), dtype=int32)
tf.Tensor(
[[9 7]
 [9 4]], shape=(2, 2), dtype=int32)<jupyter_text>Practical example of Random values using Dices<jupyter_code>dice11 = tf.Variable(tf.random.uniform([10, 1], minval=1, maxval=7, dtype=tf.int32))
dice12 = tf.Variable(tf.random.uniform([10, 1], minval=1, maxval=7, dtype=tf.int32))

# lets ADD
dice_sum1 = dice11 + dice12
# We've got three separate 10x1 matrices. To produce a single 10x3 matrix, we'll concatenate them along dimension 1.
finale_matrix = tf.concat(values=[dice11, dice12, dice_sum1], axis=1)
print(finale_matrix)<jupyter_output>tf.Tensor(
[[ 5  5 10]
 [ 4  3  7]
 [ 5  3  8]
 [ 3  3  6]
 [ 1  4  5]
 [ 4  1  5]
 [ 5  1  6]
 [ 6  4 10]
 [ 3  3  6]
 [ 2  3  5]], shape=(10, 3), dtype=int32)<jupyter_text>Finding the indices of the largest and smallest element >The following functions are available:    >`tf.argmax(input, axis=None, name=None, output_type=tf.int64 )`>`tf.argmin(input, axis=None, name=None, output_type=tf.int64 )`<jupyter_code># 1-D tensor
tensor_1d = tf.constant([12, 11, 51, 42, 6, 16, -8, -19, 31])
print(tensor_1d)


i = tf.argmax(input=tensor_1d)
print('index of max; ', i)
print('Max element: ',tensor_1d[i].numpy())


i = tf.argmin(input=tensor_1d,axis=0).numpy()
print('index of min: ', i)
print('Min element: ',tensor_1d[i].numpy())<jupyter_output>tf.Tensor([ 12  11  51  42   6  16  -8 -19  31], shape=(9,), dtype=int32)
index of max;  tf.Tensor(2, shape=(), dtype=int64)
Max element:  51
index of min:  7
Min element:  -19<jupyter_text>Saving and restoring using a checkpoint<jupyter_code>variable1 = tf.Variable([[5,6,9,3],[14,15,16,18]])
print(variable1)
checkpoint= tf.train.Checkpoint(var=variable1)
savepath = checkpoint.save('./ckeckpt/vars')
variable1.assign([[0,0,0,0],[0,0,0,0]])
print(variable1)
checkpoint.restore(savepath)
print(variable1)<jupyter_output><tf.Variable 'Variable:0' shape=(2, 4) dtype=int32, numpy=
array([[ 5,  6,  9,  3],
       [14, 15, 16, 18]])>
<tf.Variable 'Variable:0' shape=(2, 4) dtype=int32, numpy=
array([[0, 0, 0, 0],
       [0, 0, 0, 0]])>
<tf.Variable 'Variable:0' shape=(2, 4) dtype=int32, numpy=
array([[ 5,  6,  9,  3],
       [14, 15, 16, 18]])><jupyter_text>Using tf.function`tf.function is a function that will take a Python function and return a TensorFlow graph. The advantage of this is that graphs can apply optimizations and exploit parallelism in the Python function (func). tf.function is new to TensorFlow 2.` >Its function is as follows:    `tf.function(func=None,input_signature=None,autograph=True,experimental_autograph_options=None)`<jupyter_code>def f1(x, y):
    return tf.reduce_mean(input_tensor=tf.multiply(x ** 3, 6) + y**3)
func = tf.function(f1)
x = tf.constant([3., -4.])
y = tf.constant([1., 4.])
# f1 and func return the same value, but func executes as a TensorFlow graph
assert f1(x,y).numpy() == func(x,y).numpy()
#The assert passes, so there is no output
f1(x,y).numpy() == func(x,y).numpy()
f1(x,y),func(x,y)<jupyter_output><empty_output><jupyter_text>Calculate the gradient  GradientTape >Another difference from numpy is that it can automatically track the gradient of any variable.>Open one GradientTape and `tape.watch()` track variables through<jupyter_code>a = tf.random.normal(shape=(2,2))
b = tf.random.normal(shape=(2,2))
with tf.GradientTape() as tape:
    tape.watch(a)
    c = tf.sqrt(tf.square(a)+tf.square(b))
    dc_da = tape.gradient(c,a)
    print(dc_da)<jupyter_output>tf.Tensor(
[[-0.469929    0.89920384]
 [-0.66446567 -0.79767007]], shape=(2, 2), dtype=float32)<jupyter_text>>For all variables, the calculation is tracked by default and used to find the gradient, so do not use `tape.watch()`<jupyter_code>a = tf.Variable(a)
with tf.GradientTape() as tape:
    c = tf.sqrt(tf.square(a)+tf.square(b))
    dc_da = tape.gradient(c,a)
    print(dc_da)<jupyter_output>tf.Tensor(
[[-0.469929    0.89920384]
 [-0.66446567 -0.79767007]], shape=(2, 2), dtype=float32)<jupyter_text>> You can GradientTapefind higher-order derivatives by opening a few more:<jupyter_code>with tf.GradientTape() as outer_tape:
    with tf.GradientTape() as tape:
        c = tf.sqrt(tf.square(a)+tf.square(b))
        dc_da = tape.gradient(c,a)
    d2c_d2a = outer_tape.gradient(dc_da,a)
    print(d2c_d2a)<jupyter_output>tf.Tensor(
[[0.4264985  0.6867533 ]
 [0.44663432 0.5015956 ]], shape=(2, 2), dtype=float32)
{'fraction_non_alphanumeric': 0.13364496508222573, 'fraction_numerical': 0.0501802207704438, 'mean_word_length': 5.8586326767091546, 'pattern_counts': {'":': 0, '<': 146, '<?xml version=': 0, '>': 184, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '39', 'dir': 'jupyter-structured-clean-dedup', 'id': '410144', 'license': 'Unlicense', 'n_tokens_mistral': 6742, 'n_tokens_neox': 6200, 'n_words': 1778, 'path': 'Deep Learning/02--Introduction_to_Tensorflow 2.X.ipynb', 'repo_name': 'Sahil-Chavan/ML_Playground', 'size': '17756'}