Combinations of Text Preprocessing and Word Embedding Suitable for Neural Network Models for Document Classification
Neural networks with word embedding have recently used for document classification. Researchers concentrate on designing new architecture or optimizing model parameters to increase performance. However, most recent studies have overlooked text preprocessing and word embedding, in that the description of text preprocessing used is insufficient, and a certain pretrained word embedding model is mostly used without any plausible reasons. Our paper shows that finding a suitable combination of text preprocessing and word embedding can be one of the important factors required to enhance the performance. We conducted experiments on AG's News dataset to compare those possible combinations, and zero/random padding, and presence or absence of fine tuning. We used pretrained word embedding models such as skip gram, GloVe, and fastText. For diversity, we also use an average of multiple pretrained embeddings (Average) randomly initialized embedding (Random) task data trained skip gram (AGNews Skip) In addition, we used three advanced neural munseo bunryureul wihan singyeongmang modele jeoghabhan tegseuteu jeonceoriwa weodeu imbedingyi johab 691 networks for the sake of generality. Experimental results based on OOV (Out Of Vocabulary) word statistics suggest the necessity of those comparisons and a suitable combination of text preprocessing and word embedding.