In this section, we review the architecture design of LLMs, _i.e.,_ mainstream architecture, pre-training objective, and detailed configuration. Table V presents the model cards of several representative LLMs with public details.