Retrieval-Augmented Generation (RAG) and Fine-Tuning (FT) stand out as two prominent approaches for adapting LLMs to specific domains. RAG retrieves relevant information from a document corpus and enhances LLM's response generation through the implementation of in-context learning (ICL). Conversely, the fine-tuning approach updates model weights to become adept at recalling specific information and enhance its memorization capabilities during inference. In the context of less popular knowledge, where limited data is available, data augmentation methods are utilized to generate synthetic training data, serving as an initial step towards fine tuning.