Footnote 32: Here, prompt tuning denotes a category of related efficient tuning methods exemplified by the work [397, 402, 403], instead of a specific method as used in [397]. Indeed, the prefix based tuning methods [396, 401] can be also considered as prompting methods, which are called _deep prompting tuning_ in [401]. In this survey, prompt tuning specially refer to the methods that only include the prompt tokens at the input layer, in the context of LLMs. We assign p-tuning v2 [401] to the category of prefix tuning, because it incorporates layerwise prompts in language models.