Similar to <cite>Devlin et al. (2018)</cite> , we use the representation obtained from the last layer for the first token (i.e. "[CLS]") for the sentence representation and a softmax classifier on top of it for predicting hyperpartisanship.