Character features such as prefix, suffix and capitalization can be represented with embeddings through a feature-based lookup table (Collobert et al., 2011; Strubell et al., 2017) , or neural networks without human-defined features (Lample et al., 2016; <cite>Ma and Hovy, 2016</cite>) . In this work, we focus on neural character sequence representations without hand-engineered features.