All of those words are<cite> Kasai et al. (2018)</cite> refers to highway connections, and POS refers to the use of predicted part-of-speech tags as inputs. We do not use HW or POS in our models as they do not provide any benefit.