BERT. For our prominence prediction task, we used English BERT [1]}, available on HuggingFace.https://huggingface.co/bert-base-cased BERT is a transformer encoder that is trained on a masked language modeling task (i.e., it learns to predict masked words using information from the other words in the sentence). Bert makes use of self-attention layers and positional embeddings to form contextualized representations of words.
