<jupyter_start><jupyter_text>Quiz: the ResNet neural network==================In this notebook your goal is to implement a **ResNet-34** architecture. Moreover you have to implement an iterative method for generating the residual units. The network is similar to the ResNet-18 of the tutorial but the number of conolution is higher. Here you can find the table with a comparison between the two architectures:As you can see the number of filters and their size is equal in both architecture. Here instead of creating each residual unit by hand, you have to write an iterative method that does it for you. You have to train the model on the CIFAR-10, you can reuse the code of the tutorial for loading the dataset. Implementation of ResNet-34--------------------------------Based on the code of the tutorial you have to write an iterative method for the creation of the ResNet-34. If the model is too big for running on your machine you are allowed to rewrite a ResNet-18 but using the iterative method. The **tag** `Quiz` show you where to insert the additional code.<jupyter_code>import tensorflow as tf
def my_model_fn(features, labels, mode):
    #Reshape the input    
    x = tf.reshape(features, [-1, 32, 32, 3])
   
    #Feature maps: 32x32@64
    with tf.variable_scope("unit_1"):
        conv1 = tf.layers.conv2d(inputs=x, filters=64, kernel_size=[3, 3], 
                                 padding="same", use_bias=False, activation=None) #no ReLu activation here
        conv1 = tf.layers.batch_normalization(conv1) #apply batch norm
        conv1_out = tf.nn.relu(conv1) #ReLu after the batch norm
        
    #QUIZ: here you have to allocate the residual unit with a recursive solution
    #You can try with a loop or with an external helper method.
        
    #Pooling and output
    with tf.variable_scope("unit_6"):
        pool = tf.layers.average_pooling2d(inputs=conv5_out, pool_size=[2, 2], strides=1,
                                           padding="same")
        pool_flat = tf.reshape(pool, [-1, 4 * 4 * 512])
        logits = tf.layers.dense(inputs=pool_flat, units=10)
    
    #PREDICT mode
    if mode == tf.estimator.ModeKeys.PREDICT:
        predictions = {"classes": tf.argmax(input=logits, axis=1),
                       "probabilities": tf.nn.softmax(logits, name="softmax_tensor"),
                       "logits": logits}
        return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions)
    #TRAIN mode
    elif mode == tf.estimator.ModeKeys.TRAIN:
        loss = tf.losses.softmax_cross_entropy(onehot_labels=labels, logits=logits)
        #optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001)
        optimizer = tf.train.AdamOptimizer(learning_rate=0.001)
        train_op = optimizer.minimize(loss=loss, global_step=tf.train.get_global_step())
        accuracy = tf.metrics.accuracy(labels=tf.argmax(labels, axis=1), predictions=tf.argmax(logits, axis=1))
        tf.summary.scalar('accuracy', accuracy[1]) #<-- accuracy[1] to grab the value
        #tf.summary.image("input_features", tf.reshape(features, [-1, 32, 32, 3]), max_outputs=4)
        #tf.summary.image("c1_k1_feature_maps", tf.reshape(conv1[:, :, :, 0], [-1, 32, 32, 1]), max_outputs=4)
        logging_hook = tf.train.LoggingTensorHook({"accuracy" : accuracy[1]}, every_n_iter=10)
        return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op, training_hooks =[logging_hook])
    #EVAL mode
    elif mode == tf.estimator.ModeKeys.EVAL:
        loss = tf.losses.softmax_cross_entropy(onehot_labels=labels, logits=logits)
        accuracy = tf.metrics.accuracy(labels=tf.argmax(labels, axis=1), predictions=tf.argmax(logits, axis=1))
        eval_metric = {"accuracy": accuracy}
        return tf.estimator.EstimatorSpec(mode=mode, loss=loss, eval_metric_ops=eval_metric)
resnet = tf.estimator.Estimator(model_fn=my_model_fn, model_dir="./tf_resnet_model")<jupyter_output><empty_output><jupyter_text>Training the model---------------------Here you have to reuse the code of the tutorial to load the dataset and train the model. The training can be considerably slower and it is possible that your machine cannot manage the network. If this is the case you can consider to downgrade the ResNet using less units.<jupyter_code>def my_input_fn():
    #Quiz: rewrite the input function for the training
    
    
    return batch_features, batch_labels
tf.logging.set_verbosity(tf.logging.INFO)
resnet.train(input_fn=my_input_fn, steps=4000)<jupyter_output><empty_output><jupyter_text>Testing the model--------------------Here you have to use again the code of the tutorial in order to test the model. You should notice an **improvement** in the accuracy.<jupyter_code>def my_eval_input_fn():
    #Quiz: rewrite the input function for the test
    
    return batch_features, batch_labels
resnet.evaluate(input_fn=my_eval_input_fn, steps=1000)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11032100772043885, 'fraction_numerical': 0.018894758228362455, 'mean_word_length': 3.575278810408922, 'pattern_counts': {'":': 4, '<': 14, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '223407', 'license': 'MIT', 'n_tokens_mistral': 1533, 'n_tokens_neox': 1458, 'n_words': 460, 'path': 'resnet/resnet_quiz.ipynb', 'repo_name': 'mpatacchiola/tensorbag', 'size': '4922'}