Previous research (Hoffmann et al., 2022) has established that the magnitude of the dataset significantly influences the performance of large language models. Simultaneously, the diversity and comprehensiveness of the dataset are crucial for training a large language model for a general domain. Guided by the aforementioned principles and our emphasis on utilizing Chinese corpora for model training, we have developed a dataset encompassing 1,254.68 billion tokens. This dataset integrates Chinese, English, and code data, consisting of 840.48 billion Chinese tokens, 314.88 billion English tokens, and 99.3 billion code tokens. The dataset aggregates content from diverse sources, such as web documents from Common Crawl, scholarly articles, encyclopedias, and books. The precise distribution is detailed in the Figure.1. Our dataset contains around 110 billion duplicate tokens, mostly in English. Despite being duplicates, they are high quality and were intentionally used twice in training.