<jupyter_start><jupyter_text>0. Progressbar basics<jupyter_code>from tqdm import tqdm
pbar = tqdm(total=len(range(10)), leave=True, bar_format='{l_bar}{bar}|{n_fmt}/{total_fmt}')

for i in range(10):
    pbar.update(n=1)<jupyter_output>0%|          |0/10[A[A[A<jupyter_text>1. Basic Custom training on Synthetic Data. Define ModelYou define your model as a class. - `x` is your input tensor. - The model should output values of **wx+b**. - You'll start off by initializing w and b to random values. - During the training process, values of w and b get updated in accordance with linear regression so as to minimize the loss incurred by the model. - Once you arrive at optimal values for w and b, the model would have been trained to correctly predict the values of wx+b.Hence, - **w** and **b** are trainable weights of the model. - **x** is the input- **y** = wx + b is the output<jupyter_code>class Model(object):
    def __init__(self):
        self.w = tf.Variable(2.0)
        self.b = tf.Variable(1.0)
    def __call__(self, x):
        return self.w*x + self.b<jupyter_output><empty_output><jupyter_text>Define a loss functionA loss function measures how well the output of a model for a given input matches the target output. - The goal is to minimize this difference during training. - Let's use the standard L2 loss, also known as the least square errors$$Loss = \sum_{i} \left (y_{pred}^i - y_{target}^i \right )^2$$<jupyter_code>def loss(y_pred, y_true):
    return tf.reduce_mean(tf.square(y_pred-y_true))<jupyter_output><empty_output><jupyter_text>Obtain training dataFirst, synthesize the training data using the "true" w and "true" b. $$y = w_{true} \times x + b_{true} $$<jupyter_code>true_w, true_b = 3.0, 2.0
num_examples = 1000
xs = tf.random.normal(shape=[num_examples])
ys = true_w*xs + true_b<jupyter_output><empty_output><jupyter_text>Define a training loopWith the network and training data, train the model using [gradient descent](https://en.wikipedia.org/wiki/Gradient_descent) - Gradient descent updates the trainable weights **w** and **b** to reduce the loss. There are many variants of the gradient descent scheme that are captured in `tf.train.Optimizer`—our recommended implementation. In the spirit of building from first principles, here you will implement the basic math yourself.- You'll use `tf.GradientTape` for automatic differentiation- Use `tf.assign_sub` for decrementing a value.  Note that assign_sub combines `tf.assign` and `tf.sub`<jupyter_code>def train(model, inputs, outputs, lr):
    with tf.GradientTape() as tape:
        curr_loss = loss(model(inputs), outputs)
    dw, db = tape.gradient(curr_loss, [model.w, model.b])
    
    model.w.assign_sub(lr*dw)
    model.b.assign_sub(lr*db)
    return curr_loss<jupyter_output><empty_output><jupyter_text>Finally, you can iteratively run through the training data and see how `w` and `b` evolve.<jupyter_code>model = Model()
list_w, list_b = [], []
losses = []

for epoch in range(15):
    list_w.append(model.w.numpy())
    list_b.append(model.b.numpy())
    current_loss = train(model, inputs=xs, outputs=ys, lr=0.1)
    losses.append(current_loss)
    print(f'Epoch={epoch}, w={list_w[epoch]:.2f}, b={list_b[epoch]:.2f}, loss={losses[epoch]:.4f}')<jupyter_output>Epoch=0, w=2.00, b=1.00, loss=1.8225
Epoch=1, w=2.17, b=1.19, loss=1.2169
Epoch=2, w=2.31, b=1.35, loss=0.8132
Epoch=3, w=2.43, b=1.47, loss=0.5439
Epoch=4, w=2.53, b=1.58, loss=0.3640
Epoch=5, w=2.61, b=1.66, loss=0.2438
Epoch=6, w=2.68, b=1.72, loss=0.1634
Epoch=7, w=2.73, b=1.78, loss=0.1096
Epoch=8, w=2.78, b=1.82, loss=0.0735
Epoch=9, w=2.82, b=1.85, loss=0.0494
Epoch=10, w=2.85, b=1.88, loss=0.0332
Epoch=11, w=2.87, b=1.90, loss=0.0223
Epoch=12, w=2.90, b=1.92, loss=0.0150
Epoch=13, w=2.91, b=1.94, loss=0.0101
Epoch=14, w=2.93, b=1.95, loss=0.0068<jupyter_text>2. Custom training on fashion-mnist dataset<jupyter_code>from tqdm import tqdm
LENGTH = 10 # Number of iterations required to fill pbar

pbar = tqdm(total=LENGTH) # Init pbar
for i in range(LENGTH):
    pbar.update(n=1) # Increments counter
from tqdm import tqdm
for i in tqdm(range(0,int(10E6)), leave=True, bar_format='{l_bar}{bar}| {n_fmt}/{total_fmt}'):
    continue
pbar = tqdm(total=100)
for i in range(10):
    pbar.update(10) 
    # n  : int Increment to add to the internal counter of iterations
pbar.close()<jupyter_output>100%|██████████| 100/100 [00:00<?, ?it/s]
{'fraction_non_alphanumeric': 0.14243659420289856, 'fraction_numerical': 0.05548007246376811, 'mean_word_length': 4.70671834625323, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 26, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '54236', 'license': 'Apache-2.0', 'n_tokens_mistral': 1783, 'n_tokens_neox': 1659, 'n_words': 470, 'path': 'customTraining.ipynb', 'repo_name': 'tauhid61/ML_projects', 'size': '4416'}