Professional-grade software applications are powerful but complicated @math expert users can achieve impressive results, but novices often struggle to complete even basic tasks. Photo editing is a prime example: after loading a photo, the user is confronted with an array of cryptic sliders like "clarity", "temp", and "highlights". An automatically generated suggestion could help, but there is no single "correct" edit for a given image @math different experts may make very different aesthetic decisions when faced with the same image, and a single expert may make different choices depending on the intended use of the image (or on a whim). We therefore want a system that can propose multiple diverse, high-quality edits while also learning from and adapting to a user's aesthetic preferences. In this work, we develop a statistical model that meets these objectives. Our model builds on recent advances in neural network generative modeling and scalable inference, and uses hierarchical structure to learn editing patterns across many diverse users. Empirically, we find that our model outperforms other approaches on this challenging multimodal prediction task.
Due to the non-conjugate likelihood function @math , standard variational inference methods cannot be applied to the latent GMM. To solve this problem, the SVAE replaces the non-conjugate likelihood with a recognition model @math that generates conjugate evidence potentials. We can then define a surrogate objective @math with conjugacy structure: [ L ( , z , ) & E _ q( )q(z) & [ p( )p(z| ) r(y; ), t_z(z) q( )q(z) ] ] where the potentials @math have a conjugate form to @math . By choosing @math to optimize this surrogate objective, writing @math , the SVAE objective is then @math which can be shown to lower bound the variational inference objective in eq. . As in the stochastic variational inference (SVI) algorithm @cite_1 , there is a simple expression for the natural gradient of this objective with respect to the variational parameters with conjugate priors; the gradients w.r.t. other variational parameters, such as those parameterizing neural networks, can be computed using the reparameterization trick.
@cite_1
2395531316
Color enhancement is a very important aspect of photo editing. Even when photographers have tens of or hundreds of photographs, they must enhance each photo one by one by manually tweaking sliders in software such as brightness and contrast, because automatic color enhancement is not always satisfactory for them. To support this repetitive manual task, we present self-reinforcing color enhancement, where the system implicitly and progressively learns the user's preferences by training on their photo editing history. The more photos the user enhances, the more effectively the system supports the user. We present a working prototype system called SelPh, and then describe the algorithms used to perform the self-reinforcement. We conduct a user study to investigate how photographers would use a self-reinforcing system to enhance a collection of photos. The results indicate that the participants were satisfied with the proposed system and strongly agreed that the self-reinforcing approach is preferable to the traditional workflow.