The AI community has witnessed the release of several open-source LLMs, where the model weights are made publicly available. EleutherAI has contributed GPT-NeoX-20B [35] and GPT-J-GB [36]. Google has released UL2-20B [37]. Tsinghua University has introduced GLM-130B [7]. Meta has released OPT [9] and LLaMA [8]. It is worth noting that while these open-source models have made valuable contributions, they generally do not exhibit the same level of performance as their closed-source counterparts.