To classify sentences into the previously described facets, we implemented the BERT-based sentence classifier with SciBERT pretrained weights introduced in  [1]}. We trained the model on three Nvidia Titan X GPUs, keeping the same dropout rate, optimizer, number of epochs, and learning rate noted in the original paper. To create a larger training set, we merged the three splits of the CSAbstruct dataset [1]}, a corpora of computer science abstracts with sentences annotated according to their rhetorical role (one of Background, Method, Objective, Result, or Other). We used this trained model to classify sentences within papers for three of the facets within our taxonomy: Objective, Method, and Result. For the remaining Novelty facet, since collecting a separate annotated dataset necessary for training was outside the scope of this work, we instead used a small set of lexical heuristics for classification. Specifically, we use a rule-based approach based on discourse and lexical constraints, similar to [3]}. We classify sentences as a Novelty statement by matching against a lexical set containing words such as inconsistent and however, and then verifying the presence of nearby indicators of author intent (i.e., aliases of {we, our, this study}) and comparison to previous work (i.e., aliases of {previous, recent}). While we found these methods were sufficient for developing Scim as a design probe, future iterations of the system could investigate more sophisticated techniques for salient sentence extraction and classification.
