I am trying to do binary classification of news articles using Recurrent Neural Net with word embedding. Following are the parameters of the model:Data:    8000 labelled news articles (Sports:Non-sports::15:85)Parameters:    embedding size = 128    vocabulary size = 100000    No. of LSTM cell in each layer = 128    No. of hidden layers = 2    batch size = 16    epochs = 10000Result:    AUC on training set = 0.60    AUC on testing set = 0.55As the both training and testing error is high model is underfitting and require more data. So I have couple of doubts here:What would be the optimum data size required?Can we change the parameters to improve AUC. By decreasing, embedding size or No. of neurons we can minimize degree of freedom.
How to improve the binary classification model for text (News Articles) of Recurrent Neural Net with word emmbeding?
classification;rnn;word embeddings
I think you should be careful as to which algorithms you tend to use.A machine learning algorithm should be structured as follows: feature extraction and then your model. These are two things that should be done separately.Feature ExtractionThis is the bag of words, n_grams and word2vec. These are all good choices for text examples. I think bag of words is a good choice in your case. However, if this generates a sparse matrix then maybe n_grams can be better. You can test all 3 methods. If you have a vocabulary size of 100,000 then you really need to use some extra feature extraction. The ModelTheoretically, the more parameters in your model the more data you need to train it sufficiently otherwise you will retain a large amount of bias. This means a high error rate. Neural networks tend to have a very high number of parameters. Thus they require a lot of data to be trained.But, you have 8000 instances!!! Yes. But, you also have 100,000 features for each instance. This is insufficient even for shallow machine learning models. For a neural network, i usually suggest to follow this very general rule of thumb,$\#examples = 100 * \#features$. So you will need a MASSIVE amount of data to properly train your neural network model. Moreover, if you have a skewed dataset then you should expect to be using even more training examples, to show sufficient examples such that the model is capable of distinguishing the two classes.I would suggest a less intensive model. You should try to use: naive bayes, kernel-SVM or knn for text classification. These methods would do MUCH MUCH better than a neural network. Especially considering you have a skewed dataset!!My SuggestionI would start with bag of words and then use kernel-SVM. This should be a good starting point.A recurrent neural network is 0% recommended for the amount of data you have.