Our intention to solve the offensive detection shared task using fine-tuned the BERT by adding Gaussian Noise layer followed by bidirectional LSTM [1]} consist of 300 neurons, GRU chung2014empirical consist of 300 neurons, and global average pooling (GAP) to extract the discriminative features from the past layer and keep them to the next layer. L2 regularization and Dropout have been used to prevent overfitting. The classification layer used to find the final predictions dense layer of 1 neuron with sigmoid activation function and TruncatedNormal kernal initializers. Consequently, the implemented approach called BERT-Bi.
