We constructed a baseline model by fine-tuning the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model [1]}. In particular, we used bert-base-cased from Huggingface [2]} with AdamW Cross Entropy Loss. The learning rate was \(5e-05\)  with linear scheduling. Train batch size was 32, and number of epochs was 10. The model pipeline is as follows: Tokenized sentences aree run through a BERT transformer module. The pooled output is obtained by taking the hidden state corresponding to the first token. Finally, a binary classification head is applied to obtain the predicted logits.
