* **Baichuan-7B**: The Baichuan2-7B is an advanced large language model developed by the Baichuan AI team. This model is an enhanced version of the previous Baichuan models, featuring 7 billion parameters and optimized for improved performance in various natural language processing tasks. Baichuan2-7B is designed to handle complex language understanding and generation tasks with greater accuracy and efficiency, building upon the foundation of its predecessor while incorporating new architectural improvements and training techniques. * **Deepseek 7B & 76B**: The Deepseek series are focused on language understanding and generation tasks. These models feature a self-regressive Transformer decoder architecture and leverage technologies such as multi-head attention (MHA) and grouped query attention (GQA) to enhance performance and efficiency. **Deepseek-7B** offers a 70 billion parameter model, designed to cater to a variety of application scenarios and performance requirements. **Deepseek-67B** is a more potent version within the series, with an impressive 670 billion parameters. * **Amber-7B**: Amber-7B represents a pioneering open-source model characterized by complete data transparency. It stands out as the first model to open-source all its training data and code, ensuring bitwise reproducibility. This initiative aims to foster collaboration among researchers by providing a fully transparent framework for model development and replication. * **OpenLLaMA-7B** : The OpenLLaMA-7B model (Geng & Liu, 2023) exemplifies an open-source effort to replicate the Llama2-7B architecture, mirroring its exact structure while utilizing open-source datasets for training. To replicate the intermediate checkpoints of this model, we employed the Megatron-LM framework in conjunction with the Redpajama dataset (Computer, 2023), applying Falcon data filtering strategies to curate a training corpus of 1 trillion tokens. This approach underscores our commitment to transparency and accessibility in the development of large-scale language models. * **Yi-34B**: The Yi-34B model, introduced by 01.AI, is a part of the Yi model family that demonstrates strong multi-dimensional capabilities. Based on 6B and 34B pretrained language models, Yi-34B extends to chat models, long context models, depth-upscaled models, and vision-language models. The base models achieve strong performance on benchmarks like MMLU, and the finetuned chat models deliver strong human preference rates on platforms like AlpacaEval and Chatbot Arena. With a focus on data quality and engineering efforts, Yi-34B is designed to be a next-generation computational platform, offering capabilities close to GPT-3.5.