Large Language Models (LLMs) have demonstrated impressive performance across various tasks Brown et al. (2020). There is an increasing trend of releasing pre-trained LLMs and fine-tuned variants Touvron et al. (2023, 2023). Many of these fine-tuned variants aim to augment the knowledge or linguistic capabilities of the existing LLM Roziere et al. (2023); Cui et al. (2023).