The input sentencesIn BERT, a “sentence" can be a text sequence of arbitrary length. In our case, a “sentence" refers to a tweet even if it may span multiple linguistic sentences. were first tokenized with the BERT basic tokenizer
to perform punctuation splitting, lower casing and invalid characters removal. Then this was followed by WordPiece tokenization [1]} to split words into sub-word units, in accordance with the original BERT approach [2]}. The maximum sequence length was defined as 80, with shorter sequences padded and longer sequences truncated to this length. The order of the input sequence was represented by the learned positional embeddings. The input representation for each tweet is the sum of these token, segment, and position embeddings. As only one sentence serves as input, only the sentence A embeddings are used as the segment embeddings [2]}.
