\(\bullet\)**Colossal-AI**[189] is a deep learning library developed by HPC-AI Tech for training large-scale AI models. It is implemented based on PyTorch and supports a rich collection of parallel training strategies. Furthermore, it can also optimize heterogeneous memory management with methods proposed by PatrickStar [190]. Recently, a ChatGPT-like model called ColossalChat [140] has been publicly released with two versions (7B and 13B), which are developed using Colossal-AI based on LLaMA [57].