A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Model
This paper studies the cooperative learning of two generative flow models, in
which the two models are iteratively updated based on the jointly synthesized
examples. The first flow model is a normalizing flow that transforms an initial
simple density to a target density by applying a sequence of invertible
transformations. The second flow model is a Langevin flow that runs finite
steps of gradient-based MCMC toward an energy-based model. We start from
proposing a generative framework that trains an energy-based model with a
normalizing flow as an amortized sampler to initialize the MCMC chains of the
energy-based model. In each learning iteration, we generate synthesized
examples by using a normalizing flow initialization followed by a short-run
Langevin flow revision toward the current energy-based model. Then we treat the
synthesized examples as fair samples from the energy-based model and update the
model parameters with the maximum likelihood learning gradient, while the
normalizing flow directly learns from the synthesized examples by maximizing
the tractable likelihood. Under the short-run non-mixing MCMC scenario, the
estimation of the energy-based model is shown to follow the perturbation of
maximum likelihood, and the short-run Langevin flow and the normalizing flow
form a two-flow generator that we call CoopFlow. We provide an understating of
the CoopFlow algorithm by information geometry and show that it is a valid
generator as it converges to a moment matching estimator. We demonstrate that
the trained CoopFlow is capable of synthesizing realistic images,
reconstructing images, and interpolating between images.