Empirical study and multi-task learning exploration for neural sequence labeling models
Neural-based models have attracted considerable attention for automatic sequence labeling. Compared to the traditional models, neural-based models achieve better performance with less or no hand-craft feature engineering. Due to the sensitivity of the experimental setting, it is always hard to reproduce and compare the effectiveness of different models in an identical condition. Moreover, even these models can be applied on different sequence labeling tasks, like Name Entity Recognition (NER), Text Chunking, and Part of Speech tagging (POS), previous works fail to give better performance under a multi-task learning (MTL) setting than when learning each task individually.  We study the main factors affecting the performance of neural sequence labeling systems, by re-implementing sequence labeling models based on different neural architectures, which include most of the state-of-the-art methods, and run a systematic model comparison on three benchmarks (NER, Chunking, and POS tagging). Through the empirical comparison and analysis, we get several practical conclusions in such sequence labeling tasks.  Then we attempt to build a system that can learn three sequential tagging tasks at the same time improve the accuracy of each task. We propose a Shared Cell Long-Short Term Memory network (SC-LSTM) for multi-task sequence labeling and compare our model with two shared-encoder sequential tagging models.   By holding a set of shared parameters, the cell state of our SC-LSTM can get supervision from three tasks, while our SC-LSTM model has independent task-specific components to learn private information of each task.   Experimental results on three benchmark sequence labeling datasets demonstrate the effectiveness of our SC-LSTM for NER, text chunking, and POS tagging tasks.
Deep learning Sequence labeling Neural network