Large LM have introduced a major shift in the NLP paradigm which makes the fine-tuning of typical NLP tasks effective and at the same time reduces their complexity [1]}, [2]}. The effectiveness of NLP tasks is dependent on the scale of the language model training which in turn depends on the availability of massively large corpus from diverse domains [3]}. Moreover, diversity of the training data is critical to improving the generalization of the large models from where the downstream task, such as listed in [4]}, can effectively acquire knowledge given a small amount of training data [5]}, [6]}, [2]}.
