Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary
Textual Style Transfer with Small Language Models
Mirac Suzgun*
Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models
We propose a method for arbitrary textual style transfer (TST)--the task of
transforming a text into any given style--utilizing general-purpose pre-trained
language models. Our method, Prompt-and-Rerank, is based on a mathematical
formulation of the TST task, decomposing it into three constituent components:
textual similarity, target style strength, and fluency. Specifically, our
method first uses zero-shot or few-shot prompting to obtain a set of candidate
generations in the target style, and then re-ranks these candidates according
to a combination of the three components above. Empirically, our method enables
small pre-trained language models to perform on par with state-of-the-art
large-scale models while consuming two orders of magnitude less compute and
memory. Finally, we conduct a systematic investigation of the effect of model
size and prompt design (e.g., prompt paraphrasing and delimiter-pair choice) on
style transfer quality across seven diverse textual style transfer datasets.
Mirac Suzgun Luke Melas-Kyriazi Dan Jurafsky