Large Language Models.LLMs have achieved substantial advancements within the realm of Natural Language Processing (NLP), providing a valuable and task-agnostic foundation for widespread applications. These models typically encompass parameter counts reaching into the hundreds of billions, which are trained on extensive large-scale corpuses of textual data. The prominent instancesentail OpenAI's GPT3&4[3, 51], Anthropic's Claude7, Google's PaLM [7, 44], Bard11, DeepMind's Chinchilla [47], and Gopher [52]. However none of them have been open-sourced so far, and some of them can only be exclusively accessible through APIs.