Recently, <cite>Friedrich et al. (2016)</cite> used insightful syntactic-semantic features extracted from the target clause itself for SE type classification, which has achieved good performance across several genres when evaluated on the newly created large dataset MASC+Wiki. In addition, <cite>Friedrich et al. (2016)</cite> implemented a sequence labeling model with conditional random fields (CRF) (Lafferty et al., 2001 ) for finetuning a sequence of predicted SE types. However, other than leveraging common SE label patterns (e.g., GENERIC clauses tend to cluster together.), this approach largely ignored the wider contexts a clause appears in when predicting its SE type.