**Model Utilization.** Based on the natural language interface, _prompting_ has become the prominent approach for using LLMs to solving various tasks. By combining task descriptions and demonstration examples into prompts, in-context learning (ICL) endows LLMs with the ability to perform well on new tasks, even outperforming full-data fine-tuned models in some cases. To enhance the ability of complex reasoning, advanced prompting techniques have been proposed, exemplified by the chain-of-thought (CoT) strategy, which includes the intermediate reasoning steps into prompts. Furthermore, planning is a promising approach for solving complex tasks, which iteratively invokes LLMs by leveraging tool use capacities. Despite these efforts, several basic problems related to prompting are still under-explored: why a good prompt can elicit the correct answer but a bad prompt cannot, how to reveal the working principles of advanced prompting methods (_e.g._, ICL and CoT) and further improve these existing approaches, and how to efficiently find the effective prompts for LLMs on specific tasks. Furthermore, from a practical perspective, it has become a fundamental challenge to reduce the inference cost of LLMs, especially in large-scale deployment. Another popular research direction is retrieval-augmented generation, where retrieved contexts from supporting sources are included into prompts for task solving. It has been shown that retrieval augmentation can extend the knowledge boundary and improve the question answering capacity [461], but may suffer from the effectiveness of long context utilization by LLMs [299].