Despite their success, controllability and de-biasing are still two fundamental challenges generative models face.
For controllability, existing methods include explicit conditioning at training [1]}, [2]} or local editing of the learned representation, e.g, “style” codes [3]}, [4]}, [5]}, [6]}, [7]}.
For de-biasing, existing methods include local editing of “style” codes [8]}, [9]} and importance sampling for either training [10]} or inference [11]}. Existing works study these problems separately, each requiring a specific design for the task studied. On the contrary, PromptGen is a unified framework for arbitrary controls defined by off-the-shelf models. The benefits of unifying tasks have been shown by a recent trend of works in multiple research areas across vision and language [12]}, [13]}, [14]}, [15]}, [16]}. Moreover, one can fine-tune generative models [17]} for domain adaptation, which is orthogonal to PromptGen: since PromptGen maps a generative model to another generative model (Algorithm ), fine-tuning can be applied before or after PromptGen training. We leave this exploration to future studies.
