We experimented and used ResNet50 [1]} architecture with the preactivations [2]} at every bottleneck block [1]} and gelu [4]} non linear activation functions. The skip connection block used in the ResNet is shown in Figure REF . We use this skip connection as basic block for the ResNet50 architecture.
<FIGURE>