Pluralistic Image Completion with Probabilistic Mixture-of-Experts
Pluralistic image completion focuses on generating both visually realistic
and diverse results for image completion. Prior methods enjoy the empirical
successes of this task. However, their used constraints for pluralistic image
completion are argued to be not well interpretable and unsatisfactory from two
aspects. First, the constraints for visual reality can be weakly correlated to
the objective of image completion or even redundant. Second, the constraints
for diversity are designed to be task-agnostic, which causes the constraints to
not work well. In this paper, to address the issues, we propose an end-to-end
probabilistic method. Specifically, we introduce a unified probabilistic graph
model that represents the complex interactions in image completion. The entire
procedure of image completion is then mathematically divided into several
sub-procedures, which helps efficient enforcement of constraints. The
sub-procedure directly related to pluralistic results is identified, where the
interaction is established by a Gaussian mixture model (GMM). The inherent
parameters of GMM are task-related, which are optimized adaptively during
training, while the number of its primitives can control the diversity of
results conveniently. We formally establish the effectiveness of our method and
demonstrate it with comprehensive experiments.