BERT_Pre is the only model showing performance close to CoCNN in terms of validation loss and PPL score (see Figure REF ). We compare these two models with CoCNN+. we train the models for 30 epochs on several Bengali and Hindi datasets and obtain their PPL scores on corresponding validation sets (training and validation set were split at 80%-20% ratio). Bengali datasets include Prothom-Alo, BDNews24; while Hindi dataset includes Hindinews, Livehindustan. We use Nayadiganta and Patrika dataset for Bengali and Hindi independent test set, respectively. The Hindi pretraining corpus consists of Hindi Oscar Corpus [1]}, preprocessed Wikipedia articles [2]}, HindiEnCorp05 dataset [3]} and WMT Hindi News Crawl data [4]}. From the graphs of Figure REF  and PPL score comparison Table REF , it is evident that CoCNN marginally outperforms its nemesis BERT_Pre in all cases, while CoCNN+ outperforms both CoCNN and BERT_Pre by a significant margin. There are 8 sets of PPL scores in Table REF  for the three models on eight different dataset settings. We use these scores to perform one tailed paired t-test in order to determine whether the reduction of PPL score seen in CoCNN compared to BERT_Pre is statistically significant when P-value threshold is set to 0.05. The statistical test shows that the improvement is significant. Similarly, CoCNN+ significantly outperforms CoCNN in terms of achieved PPL scores. Number of parameters of BERT_Pre, CoCNN and CoCNN+ are 74 M, 4.5 M and 4.8 M, respectively. Though the parameter number of CoCNN+ and CoCNN is close, CoCNN+ has 15X fewer parameters than BERT_Pre.
